【 使用环境 】 测试环境
【 OB or 其他组件 】ob
【 使用版本 】当前最新版本
【问题描述】虚拟机宕机了(非正常关机),启动时报错,数据无所谓,我这种情况如何处理
alert 日志:
2025-11-14 21:55:09.330832|INFO|STORAGE_BLKMGR|OB_SERVER_BLOCK_MANAGER_START_SUCCESS|0|0|100754|observer|Y0-0000000000000001-0-0|start|ob_block_manager.cpp:218|"[server_start 8/18] block manager start success."
2025-11-14 21:56:19.450345|ERROR|SERVER|OB_SERVER_INSTANCE_START_FAIL|-4052|0|100754|observer|Y0-0000000000000000-0-0|start|ob_server.cpp:1173|"[server_start 9/18] observer instance start fail. you may find solutions in previous error logs or seek help from official technicians."
2025-11-14 21:56:19.481701|ERROR|SERVER|OB_SERVER_START_FAIL|-4052|0|100754|observer|Y0-0000000000000000-0-0|start|ob_server.cpp:1255|"[server_start 10/18] observer start fail, the stop status is true. you may find solutions in previous error logs or seek help from official technicians
observer日志
[2025-11-14 21:56:19.449017] INFO [LIB] recycle_tenant_allocator (ob_malloc_allocator.cpp:713) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=8] wait tenant ctx allocator success(tenant_id=1002, ctx_id=26, ctx_name="VECTOR_CTX_ID")
[2025-11-14 21:56:19.449802] INFO [LIB] recycle_tenant_allocator (ob_malloc_allocator.cpp:778) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=4] recycle tenant allocator success(tenant_id=1002)
[2025-11-14 21:56:19.449846] INFO [SERVER.OMT] create_tenant (ob_multi_tenant.cpp:1227) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=41] finish create new tenant(ret=-4052, tenant_id=1002, write_slog=false, create_step=5, bucket_lock_idx=3968)
[2025-11-14 21:56:19.449861] EDIAG [STORAGE] handle_tenant_create_commit_ (ob_server_storage_meta_replayer.cpp:172) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=14][errcode=-4052] fail to replay create tenant(ret=-4052, tenant_meta={unit:{tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"UNSH", resource:{min_cpu:1, max_cpu:4, memory_size:"7GB", log_disk_size:"18GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:1, }}, mode:1, create_timestamp:1762252562946609, is_removed:false, hidden_sys_data_disk_config_size:0, actual_data_disk_size:0}, super_block:{tenant_id:1002, replay_start_point:ObLogCursor{file_id=8, log_id=1802223, offset=56213884}, ls_meta_entry:{[ver=1,mode=0,seq=98426][2nd=22448]}, tablet_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_hidden:false, version:4, snapshot_cnt:0, preallocated_seqs:{object_seq:60000, tmp_file_seq:60000, write_seq:60000}, auto_inc_ls_epoch:0, ls_cnt:0}, create_status:1, epoch:0}) BACKTRACE:0x34e369f4 0x3566dc5c 0x34ad353c 0x34ad30ca 0x34ad7a12 0x34ad7877 0x1d879df6 0x1d8784d2 0x1d86f0e0 0x1d86da27 0x151dc174 0x10e5285f 0x10e57d32 0x297ede80 0x10e545bd 0x2b3dfbf0e628 0xb90bc5e
[2025-11-14 21:56:19.449971] EDIAG [STORAGE] apply_replay_result_ (ob_server_storage_meta_replayer.cpp:112) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=107][errcode=-4052] fail to handle tenant create commit(ret=-4052, tenant_meta={unit:{tenant_id:1002, unit_id:1001, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1001, name:"UNSH", resource:{min_cpu:1, max_cpu:4, memory_size:"7GB", log_disk_size:"18GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:1, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:1, }}, mode:1, create_timestamp:1762252562946609, is_removed:false, hidden_sys_data_disk_config_size:0, actual_data_disk_size:0}, super_block:{tenant_id:1002, replay_start_point:ObLogCursor{file_id=8, log_id=1802223, offset=56213884}, ls_meta_entry:{[ver=1,mode=0,seq=98426][2nd=22448]}, tablet_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_hidden:false, version:4, snapshot_cnt:0, preallocated_seqs:{object_seq:60000, tmp_file_seq:60000, write_seq:60000}, auto_inc_ls_epoch:0, ls_cnt:0}, create_status:1, epoch:0}) BACKTRACE:0x34e369f4 0x3566dc5c 0x34ad353c 0x34ad30ca 0x34ad7a12 0x34ad7877 0x1d879502 0x1d87853a 0x1d86f0e0 0x1d86da27 0x151dc174 0x10e5285f 0x10e57d32 0x297ede80 0x10e545bd 0x2b3dfbf0e628 0xb90bc5e
[2025-11-14 21:56:19.450049] INFO [STORAGE] apply_replay_result_ (ob_server_storage_meta_replayer.cpp:137) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=75] finish replay create tenants(ret=-4052, tenant_count=3)
[2025-11-14 21:56:19.450099] WDIAG [STORAGE] start_replay (ob_server_storage_meta_replayer.cpp:60) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4052] fail to apply repaly result(ret=-4052)
[2025-11-14 21:56:19.450139] WDIAG [STORAGE] start (ob_server_storage_meta_service.cpp:77) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=30][errcode=-4052] fail to start replayer(ret=-4052)
[2025-11-14 21:56:19.450186] INFO [STORAGE] start (ob_server_storage_meta_service.cpp:84) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=14] finish start server storage meta service(ret=-4052, cost_time_us=69670955)
[2025-11-14 21:56:19.450220] EDIAG [SERVER] start (ob_server.cpp:1018) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=28][errcode=-4052] fail to start server storage meta service(ret=-4052, ret="OB_LAST_LOG_RUINNED") BACKTRACE:0x34e369f4 0x3566dc5c 0x34ad353c 0x34ad30ca 0x34ad7a12 0x34ad7877 0x151e36d8 0x151dce09 0x10e5285f 0x10e57d32 0x297ede80 0x10e545bd 0x2b3dfbf0e628 0xb90bc5e
[2025-11-14 21:56:19.450378] ERROR [SERVER] start (ob_server.cpp:1173) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=33][errcode=-4052] [server_start 9/18] observer instance start fail. you may find solutions in previous error logs or seek help from official technicians.
[2025-11-14 21:56:19.458749] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:92) [101417][T1001_TimerWK0_ReqMemEvict][T1001][Y0-0000000000000000-0-0] [lt=32] Construct Queue Num(construct_num=0, get_push_idx()=8, get_cur_idx()=0, get_pop_idx()=0)
[2025-11-14 21:56:19.462072] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:446) [101288][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] tenant weak read service cluster heartbeat RPC fail(ret=-4076, rcode={code:-4076, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.230.200:2882", cluster_service_tablet_id={id:226})
[2025-11-14 21:56:19.462116] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:865) [101288][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=41][errcode=-4076] post cluster heartbeat rpc fail(ret=-4076, ret="OB_NEED_WAIT", tenant_id_=1, local_server_version={val:18446744073709551615, v:3}, valid_part_count=0, total_part_count=0, generate_timestamp=1763128579462066)
[2025-11-14 21:56:19.462132] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:875) [101288][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] tenant weak read service do cluster heartbeat fail(ret=-4076, ret="OB_NEED_WAIT", tenant_id_=1, last_post_cluster_heartbeat_tstamp_=1763128579362036, cluster_heartbeat_interval_=1000000, cluster_service_tablet_id={id:226}, cluster_service_master="0.0.0.0:0")
[2025-11-14 21:56:19.462153] WDIAG [STORAGE.TRANS] generate_min_weak_read_version (ob_weak_read_util.cpp:78) [101288][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4023] get gts cache error(ret=-4023, tenant_id=1)
[2025-11-14 21:56:19.462177] WDIAG [STORAGE.TRANS] generate_server_version (ob_tenant_weak_read_service.cpp:311) [101288][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=23][errcode=-4023] generate min weak read version error(ret=-4023, tenant_id=1)
[2025-11-14 21:56:19.462187] WDIAG [STORAGE.TRANS] generate_tenant_weak_read_timestamp_ (ob_tenant_weak_read_service.cpp:593) [101288][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4023] generate server version for tenant fail(ret=-4023, ret="OB_EAGAIN", tenant_id=1, index=0x2b3dfc02f1e0, server_version_epoch_tstamp_=1763128579462141)
[2025-11-14 21:56:19.481435] INFO destroy_tg (thread_mgr.cpp:89) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=24] destroy tg(tg_id=283, tg=0x2b3e1418fcb0, tg->attr_={name:StartupAccelHandler, type:4})
[2025-11-14 21:56:19.481572] EDIAG [SERVER] start (ob_server.cpp:1251) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=37][errcode=-4052] failure occurs, try to set stop and wait(ret=-4052, ret="OB_LAST_LOG_RUINNED") BACKTRACE:0x34e369f4 0x3566dc5c 0x34ad353c 0x34ad30ca 0x34ad7a12 0x34ad7877 0x151e779e 0x151dfc99 0x10e5285f 0x10e57d32 0x297ede80 0x10e545bd 0x2b3dfbf0e628 0xb90bc5e
[2025-11-14 21:56:19.481847] ERROR [SERVER] start (ob_server.cpp:1255) [100754][observer][T0][Y0-0000000000000000-0-0] [lt=146][errcode=-4052] [server_start 10/18] observer start fail, the stop status is true. you may find solutions in previous error logs or seek help from official technicians.