OB4.2.1.10重启服务器以后无法启动

【 使用环境 】测试环境
【 OB or 其他组件 】OB 4.2.1.10
【 使用版本 】
【问题描述】OB4.2.1.10重启服务器以后无法启动
【复现路径】问题出现前后相关操作
【附件及日志】推荐使用OceanBase敏捷诊断工具obdiag收集诊断信息,详情参见链接(右键跳转查看):

【SOP系列 22 】——故障诊断第一步(自助诊断和诊断信息收集)

[2025-05-21 11:01:18.445019] INFO [SERVER] destroy (ob_server.cpp:796) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] [OBSERVER_NOTICE] destroy observer end
[2025-05-21 11:01:18.445026] ERROR issue_dba_error (ob_log.cpp:1875) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-4103, file=“ob_server.cpp”, line_no=509, info="[OBSERVER_NOTICE] fail to init observer")
[2025-05-21 11:01:18.445039] EDIAG [SERVER] init (ob_server.cpp:509) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-4103] [OBSERVER_NOTICE] fail to init observer(ret=-4103, ret=“OB_CHECKSUM_ERROR”) BACKTRACE:0x125fd785 0x50b6335 0x51ad5ed 0x51ad0af 0x519a224 0x51acf53 0xa7c62ca 0xa7b7ccb 0x7306344 0x7fc722db4d7a 0x5329d6e
[2025-05-21 11:01:18.445066] ERROR init (ob_server.cpp:510) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=26][errcode=-4393] observer start process failure(msg=“observer init() has failure”, ret=-4103, ret=“OB_CHECKSUM_ERROR”)
[2025-05-21 11:01:18.445081] ERROR issue_dba_error (ob_log.cpp:1875) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-4103, file=“main.cpp”, line_no=585, info=“observer init fail”)
[2025-05-21 11:01:18.445090] EDIAG [SERVER] main (main.cpp:585) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4103] observer init fail(ret=-4103) BACKTRACE:0x125fd785 0x50b6335 0x50b4e08 0x50b484f 0x50b478c 0x50b45bd 0x7307599 0x7306691 0x7fc722db4d7a 0x5329d6e
============= [BEFORE_DESTROY] begin to show unstopped thread =============
[BEFORE_DESTROY] detect unstopped thread, tid: 2932, name: observer
============= [BEFORE_DESTROY] finish to show unstopped thread =============
[2025-05-21 11:01:18.445232] INFO [SERVER] destroy (ob_server.cpp:530) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=35] [OBSERVER_NOTICE] destroy observer begin
[2025-05-21 11:01:18.445238] INFO [SERVER] destroy (ob_server.cpp:532) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] begin to destroy config manager
[2025-05-21 11:01:18.445243] INFO [SERVER] destroy (ob_server.cpp:534) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy config manager success
[2025-05-21 11:01:18.445266] INFO [SERVER] main (main.cpp:598) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=3] observer exits(observer_version=“OceanBase_CE 4.2.1.10”)
============= [AFTER_DESTROY] begin to show unstopped thread =============
[AFTER_DESTROY] detect unstopped thread, tid: 2932, name: observer
============= [AFTER_DESTROY] finish to show unstopped thread =============
[2025-05-21 11:01:18.445359] INFO [COMMON] destroy (ob_kvcache_hazard_version.cpp:213) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=11] Hazard version begin to destroy
[2025-05-21 11:01:18.445426] WDIAG foreach_refactored (ob_hashtable.h:1457) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=17][errcode=0] hashtable not init
[2025-05-21 11:01:18.445442] WDIAG foreach_refactored (ob_hashtable.h:1457) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=15][errcode=0] hashtable not init
[2025-05-21 11:01:18.445595] WDIAG [COMMON] wrlock_all (ob_bucket_lock.cpp:229) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4006] The ObBucketLock has not been inited, (ret=-4006)
[2025-05-21 11:01:18.445609] WDIAG [COMMON] ObBucketWLockAllGuard (ob_bucket_lock.h:166) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=14][errcode=-4006] Fail to try write lock all buckets(ret=-4006)
[2025-05-21 11:01:18.445641] WDIAG begin (ob_hashtable.h:914) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4006] hashtable not init, backtrace=0x125fd785 0xee85476 0xee30688 0xee30329 0x7fc722dcc597 0x7fc722dcc73a 0x7fc722db4d81 0x5329d6e
[2025-05-21 11:01:18.447833] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=11] destroy tg(tg_id=0, tg=0x7fc722405ea0, tg->attr_={name:test1, type:3})
[2025-05-21 11:01:18.447873] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=37] destroy tg(tg_id=1, tg=0x7fc722490030, tg->attr_={name:test2, type:4})
[2025-05-21 11:01:18.447881] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=7] destroy tg(tg_id=2, tg=0x7fc722490230, tg->attr_={name:test3, type:5})
[2025-05-21 11:01:18.447888] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=3, tg=0x7fc722491950, tg->attr_={name:test4, type:2})
[2025-05-21 11:01:18.447895] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=4, tg=0x7fc722492030, tg->attr_={name:test5, type:6})
[2025-05-21 11:01:18.447901] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=5, tg=0x7fc722494030, tg->attr_={name:test6, type:7})
[2025-05-21 11:01:18.447908] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=6, tg=0x7fc722492b70, tg->attr_={name:test7, type:4})
[2025-05-21 11:01:18.447914] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=7, tg=0x7fc722492d70, tg->attr_={name:test8, type:1})
[2025-05-21 11:01:18.447920] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=8, tg=0x7fc722492eb0, tg->attr_={name:memDump, type:2})
[2025-05-21 11:01:18.447927] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=9, tg=0x7fc7224ac030, tg->attr_={name:SchemaRefTask, type:5})
[2025-05-21 11:01:18.447946] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=18] destroy tg(tg_id=10, tg=0x7fc7224ad750, tg->attr_={name:ReqMemEvict, type:3})
[2025-05-21 11:01:18.447952] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=11, tg=0x7fc7224ad8c0, tg->attr_={name:replica_control, type:2})
[2025-05-21 11:01:18.447957] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=12, tg=0x7fc7224ad980, tg->attr_={name:testObTh, type:2})
[2025-05-21 11:01:18.447963] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=13, tg=0x7fc7224ada40, tg->attr_={name:ComTh, type:2})
[2025-05-21 11:01:18.447970] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=14, tg=0x7fc7224adb00, tg->attr_={name:ComQueueTh, type:4})
[2025-05-21 11:01:18.447976] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=15, tg=0x7fc7224add00, tg->attr_={name:ComTimerTh, type:3})
[2025-05-21 11:01:18.447981] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=16, tg=0x7fc7224ade70, tg->attr_={name:Blacklist, type:2})
[2025-05-21 11:01:18.447987] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=17, tg=0x7fc7224adf30, tg->attr_={name:PartSerMigRetryQt, type:2})
[2025-05-21 11:01:18.447992] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=18, tg=0x7fc722491a10, tg->attr_={name:TransMigrate, type:4})
[2025-05-21 11:01:18.447998] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=19, tg=0x7fc722491c10, tg->attr_={name:StandbyTimestampService, type:2})
[2025-05-21 11:01:18.448004] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=20, tg=0x7fc722491cd0, tg->attr_={name:WeakRdSrv, type:2})
[2025-05-21 11:01:18.448009] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=21, tg=0x7fc722491d90, tg->attr_={name:TransTaskWork, type:4})
[2025-05-21 11:01:18.448015] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=22, tg=0x7fc72242fea0, tg->attr_={name:DDLTaskExecutor3, type:2})
[2025-05-21 11:01:18.448021] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=23, tg=0x7fc722492f70, tg->attr_={name:TSWorker, type:4})
[2025-05-21 11:01:18.448027] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=25, tg=0x7fc722493230, tg->attr_={name:RLMGR, type:2})
[2025-05-21 11:01:18.448033] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=26, tg=0x7fc7224932f0, tg->attr_={name:LeaseQueueTh, type:2})
[2025-05-21 11:01:18.448038] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=27, tg=0x7fc7224933b0, tg->attr_={name:DDLQueueTh, type:2})
[2025-05-21 11:01:18.448043] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=28, tg=0x7fc722493470, tg->attr_={name:MysqlQueueTh, type:2})
[2025-05-21 11:01:18.448049] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=29, tg=0x7fc722493530, tg->attr_={name:DDLPQueueTh, type:2})
[2025-05-21 11:01:18.448054] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=30, tg=0x7fc7224935f0, tg->attr_={name:DiagnoseQueueTh, type:2})
[2025-05-21 11:01:18.448065] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=10] destroy tg(tg_id=31, tg=0x7fc7224ae030, tg->attr_={name:DdlBuild, type:6})
[2025-05-21 11:01:18.448071] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=32, tg=0x7fc7224aeb70, tg->attr_={name:LSService, type:1})
[2025-05-21 11:01:18.448076] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=33, tg=0x7fc7224aecb0, tg->attr_={name:ObCreateStandbyFromNetActor, type:1})
[2025-05-21 11:01:18.448082] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=34, tg=0x7fc7224aedf0, tg->attr_={name:SimpleLSService, type:1})
[2025-05-21 11:01:18.448088] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=35, tg=0x7fc7224aef30, tg->attr_={name:IntermResGC, type:3})
[2025-05-21 11:01:18.448093] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=40, tg=0x7fc7224af660, tg->attr_={name:RSqlPool, type:3})
[2025-05-21 11:01:18.448099] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=41, tg=0x7fc7224af7d0, tg->attr_={name:KVCacheWash, type:3})
[2025-05-21 11:01:18.448104] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=42, tg=0x7fc7224af940, tg->attr_={name:KVCacheRep, type:3})
[2025-05-21 11:01:18.448110] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=43, tg=0x7fc7224afab0, tg->attr_={name:ObHeartbeat, type:3})
[2025-05-21 11:01:18.448115] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=44, tg=0x7fc7224afc20, tg->attr_={name:PlanCacheEvict, type:3})
[2025-05-21 11:01:18.448120] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=45, tg=0x7fc7224afd90, tg->attr_={name:TabletStatRpt, type:3})
[2025-05-21 11:01:18.448126] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=46, tg=0x7fc722431ea0, tg->attr_={name:PsCacheEvict, type:3})
[2025-05-21 11:01:18.448132] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=47, tg=0x7fc722433ea0, tg->attr_={name:MergeLoop, type:3})
[2025-05-21 11:01:18.448138] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=48, tg=0x7fc722467e40, tg->attr_={name:SSTableGC, type:3})
[2025-05-21 11:01:18.448144] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=49, tg=0x7fc722469e40, tg->attr_={name:MediumLoop, type:3})
[2025-05-21 11:01:18.448149] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=50, tg=0x7fc72246be40, tg->attr_={name:WriteCkpt, type:3})
[2025-05-21 11:01:18.448155] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=51, tg=0x7fc72246de40, tg->attr_={name:EXTLogWash, type:3})
[2025-05-21 11:01:18.448161] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=52, tg=0x7fc72246fe40, tg->attr_={name:LineCache, type:3})
[2025-05-21 11:01:18.448167] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=53, tg=0x7fc722471e40, tg->attr_={name:LocalityReload, type:3})
[2025-05-21 11:01:18.448172] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=54, tg=0x7fc7224936b0, tg->attr_={name:MemstoreGC, type:3})
[2025-05-21 11:01:18.448179] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=56, tg=0x7fc722493990, tg->attr_={name:CLOGReqMinor, type:3})
[2025-05-21 11:01:18.448201] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=22] destroy tg(tg_id=57, tg=0x7fc722493b00, tg->attr_={name:PGArchiveLog, type:3})
[2025-05-21 11:01:18.448207] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=58, tg=0x7fc722493c70, tg->attr_={name:CKPTLogRep, type:3})
[2025-05-21 11:01:18.448213] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=59, tg=0x7fc722493de0, tg->attr_={name:RebuildRetry, type:3})
[2025-05-21 11:01:18.448218] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=60, tg=0x7fc7224b0030, tg->attr_={name:TableMgrGC, type:3})
[2025-05-21 11:01:18.448223] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=61, tg=0x7fc7224b01a0, tg->attr_={name:IndexSche, type:3})
[2025-05-21 11:01:18.448229] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=62, tg=0x7fc7224b0310, tg->attr_={name:FreInfoReload, type:3})
[2025-05-21 11:01:18.448233] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=63, tg=0x7fc7224b0480, tg->attr_={name:HAGtsMgr, type:3})
[2025-05-21 11:01:18.448239] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=64, tg=0x7fc7224b05f0, tg->attr_={name:HAGtsHB, type:3})
[2025-05-21 11:01:18.448243] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=65, tg=0x7fc7224b0760, tg->attr_={name:RebuildTask, type:3})
[2025-05-21 11:01:18.448248] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=66, tg=0x7fc7224b08d0, tg->attr_={name:LogDiskMon, type:3})
[2025-05-21 11:01:18.448254] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=67, tg=0x7fc7224b0a40, tg->attr_={name:ILOGFlush, type:3})
[2025-05-21 11:01:18.448259] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=68, tg=0x7fc7224b0bb0, tg->attr_={name:ILOGPurge, type:3})
[2025-05-21 11:01:18.448264] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=69, tg=0x7fc7224b0d20, tg->attr_={name:RLogClrCache, type:3})
[2025-05-21 11:01:18.448269] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=70, tg=0x7fc7224b0e90, tg->attr_={name:TableStatRpt, type:3})
[2025-05-21 11:01:18.448274] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=71, tg=0x7fc7224b1000, tg->attr_={name:MacroMetaMgr, type:3})
[2025-05-21 11:01:18.448279] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=72, tg=0x7fc7224b1170, tg->attr_={name:StoreFileGC, type:3})
[2025-05-21 11:01:18.448285] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=73, tg=0x7fc7224b12e0, tg->attr_={name:LeaseHB, type:3})
[2025-05-21 11:01:18.448291] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=74, tg=0x7fc7224b1450, tg->attr_={name:ClusterTimer, type:3})
[2025-05-21 11:01:18.448297] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=75, tg=0x7fc7224b15c0, tg->attr_={name:MergeTimer, type:3})
[2025-05-21 11:01:18.448302] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=76, tg=0x7fc7224b1730, tg->attr_={name:CFC, type:3})
[2025-05-21 11:01:18.448308] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=77, tg=0x7fc7224b18a0, tg->attr_={name:CCDF, type:3})
[2025-05-21 11:01:18.448318] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=10] destroy tg(tg_id=78, tg=0x7fc7224b1a10, tg->attr_={name:LogMysqlPool, type:3})
[2025-05-21 11:01:18.448324] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=79, tg=0x7fc7224b1b80, tg->attr_={name:TblCliSqlPool, type:3})
[2025-05-21 11:01:18.448329] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=80, tg=0x7fc7224b1cf0, tg->attr_={name:QueryExecCtxGC, type:2})
[2025-05-21 11:01:18.448334] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=81, tg=0x7fc7224b1db0, tg->attr_={name:DtlDfc, type:3})
[2025-05-21 11:01:18.448340] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=82, tg=0x7fc7224b2030, tg->attr_={name:LogIOCb, type:4})
[2025-05-21 11:01:18.448346] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=83, tg=0x7fc7224b2230, tg->attr_={name:LogSharedQueueThread, type:4})
[2025-05-21 11:01:18.448352] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=84, tg=0x7fc7224b2430, tg->attr_={name:ReplaySrv, type:4})
[2025-05-21 11:01:18.448357] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=85, tg=0x7fc7224b2630, tg->attr_={name:LogRouteSrv, type:4})
[2025-05-21 11:01:18.448362] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=86, tg=0x7fc7224b2830, tg->attr_={name:LogRouterTimer, type:3})
[2025-05-21 11:01:18.448368] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=87, tg=0x7fc7224b4030, tg->attr_={name:LSWorker, type:7})
[2025-05-21 11:01:18.448373] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=88, tg=0x7fc7224cc030, tg->attr_={name:LSIdlePool, type:7})
[2025-05-21 11:01:18.448378] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=89, tg=0x7fc7224e4030, tg->attr_={name:LSDeadPool, type:7})
[2025-05-21 11:01:18.448384] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=90, tg=0x7fc7224b29a0, tg->attr_={name:LSTimer, type:3})
[2025-05-21 11:01:18.448389] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=91, tg=0x7fc7224b2b10, tg->attr_={name:PalfGC, type:3})
[2025-05-21 11:01:18.448395] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=92, tg=0x7fc7224b2c80, tg->attr_={name:LSFreeze, type:4})
[2025-05-21 11:01:18.448399] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=93, tg=0x7fc7224b2e80, tg->attr_={name:FetchLog, type:4})
[2025-05-21 11:01:18.448405] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=94, tg=0x7fc7224b3080, tg->attr_={name:DagScheduler, type:2})
[2025-05-21 11:01:18.448411] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=95, tg=0x7fc7224b3140, tg->attr_={name:DagWorker, type:2})
[2025-05-21 11:01:18.448416] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=96, tg=0x7fc7224b3200, tg->attr_={name:RCSrv, type:4})
[2025-05-21 11:01:18.448422] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=97, tg=0x7fc7224b3400, tg->attr_={name:ApplySrv, type:4})
[2025-05-21 11:01:18.448427] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=99, tg=0x7fc7224b3770, tg->attr_={name:StorageLogWriter, type:2})
[2025-05-21 11:01:18.448442] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=14] destroy tg(tg_id=100, tg=0x7fc7224b3830, tg->attr_={name:ReplayProcessStat, type:3})
[2025-05-21 11:01:18.448447] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=103, tg=0x7fc7224b3c80, tg->attr_={name:DDLScanTask, type:3})
[2025-05-21 11:01:18.448453] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=104, tg=0x7fc7224b3df0, tg->attr_={name:LSMetaCh, type:3})
[2025-05-21 11:01:18.448459] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=105, tg=0x7fc7224fc030, tg->attr_={name:TbMetaCh, type:3})
[2025-05-21 11:01:18.448464] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=106, tg=0x7fc7224fc1a0, tg->attr_={name:SvrMetaCh, type:3})
[2025-05-21 11:01:18.448469] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=107, tg=0x7fc7224fc310, tg->attr_={name:ArbGCTimerP, type:3})
[2025-05-21 11:01:18.448474] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=108, tg=0x7fc7224fc480, tg->attr_={name:DataDictTimer, type:3})
[2025-05-21 11:01:18.448480] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=109, tg=0x7fc7224fc5f0, tg->attr_={name:CDCSrv, type:2})
[2025-05-21 11:01:18.448486] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=110, tg=0x7fc7224fc6b0, tg->attr_={name:LogUpdater, type:3})
[2025-05-21 11:01:18.448490] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=111, tg=0x7fc7224fc820, tg->attr_={name:HeartBeatCheckTask, type:3})
[2025-05-21 11:01:18.448495] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=114, tg=0x7fc7224fcc70, tg->attr_={name:SSTableDefragment, type:3})
[2025-05-21 11:01:18.448500] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=115, tg=0x7fc7224fcde0, tg->attr_={name:TenantMetaMemMgr, type:3})
[2025-05-21 11:01:18.448506] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=116, tg=0x7fc7224fcf50, tg->attr_={name:IngressService, type:3})
[2025-05-21 11:01:18.448511] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=117, tg=0x7fc7224fd0c0, tg->attr_={name:HeartbeatService, type:1})
[2025-05-21 11:01:18.448516] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=118, tg=0x7fc7224fd200, tg->attr_={name:DetectManager, type:2})
[2025-05-21 11:01:18.448520] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=120, tg=0x7fc7224fd430, tg->attr_={name:IO_TUNING, type:2})
[2025-05-21 11:01:18.448526] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=121, tg=0x7fc7224fd4f0, tg->attr_={name:IO_SCHEDULE, type:2})
[2025-05-21 11:01:18.448531] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=122, tg=0x7fc7224fd5b0, tg->attr_={name:IO_CALLBACK, type:2})
[2025-05-21 11:01:18.448536] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=123, tg=0x7fc7224fd670, tg->attr_={name:IO_CHANNEL, type:2})
[2025-05-21 11:01:18.448541] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=124, tg=0x7fc7224fd730, tg->attr_={name:IO_HEALTH, type:4})
[2025-05-21 11:01:18.448546] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=125, tg=0x7fc7224fd930, tg->attr_={name:IO_BENCHMARK, type:2})
[2025-05-21 11:01:18.448567] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=20] destroy tg(tg_id=127, tg=0x7fc7224fdb60, tg->attr_={name:MasterKeyMgr, type:4})
[2025-05-21 11:01:18.448573] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=128, tg=0x7fc7224fdd60, tg->attr_={name:SrsMgr, type:3})
[2025-05-21 11:01:18.448579] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=129, tg=0x7fc7224fe030, tg->attr_={name:InfoPoolResize, type:3})
[2025-05-21 11:01:18.448584] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=130, tg=0x7fc7224fe1a0, tg->attr_={name:MinorScan, type:3})
[2025-05-21 11:01:18.448590] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=131, tg=0x7fc7224fe310, tg->attr_={name:MajorScan, type:3})
[2025-05-21 11:01:18.448595] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=132, tg=0x7fc7224fe480, tg->attr_={name:TransferSrv, type:1})
[2025-05-21 11:01:18.448601] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=133, tg=0x7fc7224fe5c0, tg->attr_={name:WrTimer, type:3})
[2025-05-21 11:01:18.448606] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4] destroy tg(tg_id=134, tg=0x7fc7224fe730, tg->attr_={name:SvrStartupHandler, type:4})
[2025-05-21 11:01:18.448611] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=135, tg=0x7fc7224fe930, tg->attr_={name:TTLManager, type:3})
[2025-05-21 11:01:18.448617] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=6] destroy tg(tg_id=136, tg=0x7fc7224feaa0, tg->attr_={name:TTLTabletMgr, type:3})
[2025-05-21 11:01:18.448623] INFO destroy_tg (thread_mgr.cpp:91) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy tg(tg_id=137, tg=0x7fc7224fec10, tg->attr_={name:TntSharedTimer, type:3})
[2025-05-21 11:01:18.448980] INFO [SERVER] destroy (ob_server.cpp:530) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] [OBSERVER_NOTICE] destroy observer begin
[2025-05-21 11:01:18.448995] INFO [SERVER] destroy (ob_server.cpp:532) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=15] begin to destroy config manager
[2025-05-21 11:01:18.449001] INFO [SERVER] destroy (ob_server.cpp:534) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=5] destroy config manager success
[2025-05-21 11:01:18.449006] WDIAG [CLOG] destroy (ob_server_log_block_mgr.cpp:125) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=4][errcode=0] ObServerLogBlockMgr destroy(this={dir::"", dir_fd:-1, meta_fd:-1, log_pool_meta:{curr_total_size:0, next_total_size:0, status:0}, min_block_id:0, max_block_id:0, min_log_disk_size_for_all_tenants_:0, is_inited:false})
[2025-05-21 11:01:18.449281] INFO [STORAGE.TRANS] destroy (ob_weak_read_service.cpp:54) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=12] [WRS] weak read service begin destroy
[2025-05-21 11:01:18.449295] INFO [STORAGE.TRANS] destroy (ob_weak_read_service.cpp:60) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=14] [WRS] weak read service destroy succ
[2025-05-21 11:01:18.449432] WDIAG begin (ob_hashtable.h:914) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4006] hashtable not init, backtrace=0x125fd785 0x8a187f4 0x8a8f88e 0x8bc7563 0xa7afcc9 0x7fc722dcc597 0x7fc722dcc73a 0x7fc722db4d81 0x5329d6e
[2025-05-21 11:01:18.449462] WDIAG begin (ob_hashtable.h:914) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4006] hashtable not init, backtrace=0x125fd785 0x8b9699a 0x8a8f947 0x8bc7563 0xa7afcc9 0x7fc722dcc597 0x7fc722dcc73a 0x7fc722db4d81 0x5329d6e
[2025-05-21 11:01:18.449490] WDIAG begin (ob_hashtable.h:914) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-4006] hashtable not init, backtrace=0x125fd785 0x8b96ae6 0x8a8fa07 0x8bc7563 0xa7afcc9 0x7fc722dcc597 0x7fc722dcc73a 0x7fc722db4d81 0x5329d6e
[2025-05-21 11:01:18.449516] WDIAG begin (ob_hashtable.h:914) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4006] hashtable not init, backtrace=0x125fd785 0x8a187f4 0x8a8fade 0x8bc7563 0xa7afcc9 0x7fc722dcc597 0x7fc722dcc73a 0x7fc722db4d81 0x5329d6e
[2025-05-21 11:01:18.449536] WDIAG begin (ob_hashtable.h:914) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4006] hashtable not init, backtrace=0x125fd785 0x8b96c32 0x8a8fb97 0x8bc7563 0xa7afcc9 0x7fc722dcc597 0x7fc722dcc73a 0x7fc722db4d81 0x5329d6e
[2025-05-21 11:01:18.449722] INFO [SHARE.LOCATION] destroy (ob_tablet_location_refresh_service.cpp:333) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=14] [REFRESH_TABLET_LOCATION] destroy service begin
[2025-05-21 11:01:18.449742] INFO [SHARE.LOCATION] stop (ob_tablet_location_refresh_service.cpp:317) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=19] [REFRESH_TABLET_LOCATION] stop service begin
[2025-05-21 11:01:18.449751] WDIAG [SHARE] logical_stop (ob_reentrant_thread.cpp:103) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4006] not init(ret=-4006)
[2025-05-21 11:01:18.449762] INFO [SHARE.LOCATION] stop (ob_tablet_location_refresh_service.cpp:320) [2932][observer][T0][Y0-0000000000000000-0-0] [lt=9] [REFRESH_TABLET_LOCATION] stop service end

【备注】基于 LLM 和开源文档 RAG 的论坛小助手已开放测试,在发帖时输入 [@论坛小助手] 即可召唤小助手,欢迎试用!

1 个赞

日志采集场景: 数据恢复问题
日志采集命令: obdiag gather scene run --scene=observer.recovery

推荐使用 obdiag 工具进行日志采集,并且将采集到的日志提供给我们以便确认问题所在。根据您描述的情况,服务器重启后 OceanBase 无法启动并报错 -4103 (OB_CHECKSUM_ERROR),这可能与数据文件的完整性校验有关。通过执行上述命令可以收集相关的日志信息以帮助我们进一步分析问题。

另外,为了确定具体的问题,在此需要您提供一些信息:

  1. 是否在服务器重启前对 OceanBase 集群进行了正常的关闭操作?
  2. 是否存在硬盘故障或 IO 异常的情况?
  3. 在发生问题之前,是否有过大规模的数据写入或删除操作?
  4. 能否提供完整的错误日志内容(包括 checksum error 的上下文信息)?
  5. 是否尝试过重新启动 OceanBase 集群,结果如何?

附上敏捷诊断工具 obdiag 使用帮助链接

1 个赞

感觉就是observer进程初始化阶段出问题了

断电重启的吗?OB_CHECKSUM_ERROR @论坛小助手

补充一个信息:
机器是PVE虚拟机,在尝试通过修改cpu类型为host获得avx并重启以后无法启动OB

根据您提供的日志信息,OceanBase 数据库在重启后无法启动,并且报错 OB_CHECKSUM_ERROR。以下是可能的原因和解决方法:

1. 问题分析

  • 错误码 -4103 对应的是 OB_CHECKSUM_ERROR,这通常表示数据文件的校验和不匹配,可能是由于数据文件损坏或磁盘问题导致。
  • 日志中提到 fail to init observer,说明 Observer 在初始化阶段失败,未能成功加载数据。

2. 可能原因

  • 断电或其他异常关机:如果服务器经历了断电或强制关机,可能导致数据文件未正确写入磁盘,从而引发校验和错误。
  • 磁盘故障:存储 OceanBase 数据的磁盘可能存在硬件故障或文件系统损坏。
  • 数据文件损坏:某些数据文件可能因其他原因(如 IO 错误)而损坏。

3. 解决步骤

步骤 1: 检查磁盘状态

  • 确保存储 OceanBase 数据的磁盘正常工作。
  • 使用以下命令检查磁盘健康状态:
    smartctl -a /dev/sdX
    
    (将 /dev/sdX 替换为实际的磁盘设备名)

步骤 2: 检查文件系统完整性

  • 如果怀疑文件系统损坏,可以使用 fsck 工具检查并修复文件系统:
    fsck /dev/sdX
    

步骤 3: 使用 obdiag 收集诊断信息

  • 根据您提供的信息,推荐使用 OceanBase 的诊断工具 obdiag 收集更多信息以帮助定位问题:
    obdiag collect_log -t all -d <data_dir>
    
    (将 <data_dir> 替换为实际的数据目录路径)

步骤 4: 尝试修复数据文件

  • 如果确认是数据文件损坏,可以尝试以下方法:
    1. 备份数据:在任何操作之前,请确保对现有数据进行完整备份。
    2. 使用 ob_admin 工具:尝试使用 ob_admin 工具修复数据文件:
      ob_admin -i <ip> -P <port> -u <user> repair_data
      
      (请根据实际情况替换 <ip><port><user>

步骤 5: 重新初始化 Observer

  • 如果以上方法无效,可以尝试删除数据目录并重新初始化 Observer:
    rm -rf <data_dir>
    obd cluster deploy <cluster_name> -c <config_file>
    
    (请注意,此操作会导致数据丢失,请仅在必要时执行)

4. 预防措施

  • 避免断电或强制关机,确保服务器电源稳定。
  • 定期备份数据,以便在发生类似问题时能够快速恢复。
  • 监控磁盘健康状态,及时发现并更换故障磁盘。

5. 联系支持

  • 如果上述方法均无法解决问题,建议联系 OceanBase 的技术支持团队获取进一步帮助。

希望以上信息能帮助您解决问题!如果还有其他疑问,请随时提问。

检索到的文档如下:

  1. 0001 ~ 3999
  2. SYS 租户/RS 服务问题

修改回原本的KVM64选项又提示:cpu does not support avx, Please change the server.

根据您的描述,问题的核心在于虚拟机的 CPU 配置与 OceanBase 的运行需求不匹配。以下是详细的分析和解决方法:


1. 问题分析

  • AVX 支持问题:OceanBase 社区版从某些版本开始(如 4.x 系列),对 AVX 指令集的支持有明确要求。如果虚拟机的 CPU 配置未启用 AVX 指令集,OceanBase 将无法正常启动,并提示 cpu does not support avx
  • PVE 虚拟化环境限制:Proxmox VE(PVE)默认可能不会将宿主机的所有 CPU 特性完全暴露给虚拟机,尤其是高级指令集(如 AVX、AVX2)。这可能导致即使宿主机支持 AVX,虚拟机内部却检测不到。

2. 解决方法

方法 1: 修改虚拟机配置以启用 AVX

  • 步骤 1: 登录 PVE 管理界面,进入虚拟机设置。
  • 步骤 2: 在虚拟机的 CPU 设置中,选择 host 模式或手动勾选支持的 CPU 特性。
    • 如果选择 host 模式,请确保宿主机确实支持 AVX。
    • 如果手动选择特性,请勾选 avxavx2(如果宿主机支持)。
  • 步骤 3: 保存配置并重启虚拟机。

注意:如果宿主机本身不支持 AVX,即使选择 host 模式也无法解决问题。

方法 2: 使用兼容模式部署 OceanBase

  • 如果无法更改虚拟机的 CPU 配置,可以尝试使用较低版本的 OceanBase(如 3.x 系列),这些版本对 AVX 的依赖较少。
  • 下载并部署 OceanBase 3.x 社区版:
    obd mirror add oceanbase-ce-3.x https://mirrors.oceanbase.com/obce-3.x/
    obd cluster deploy <cluster_name> -c <config_file> --mirror oceanbase-ce-3.x
    

方法 3: 更换宿主机或虚拟化平台

  • 如果当前宿主机或 PVE 环境无法满足 OceanBase 的硬件要求,建议更换到支持 AVX 的物理服务器或虚拟化平台。
  • 推荐使用支持 AVX 的云服务提供商(如 AWS、阿里云等),并在创建实例时选择支持 AVX 的实例类型。

3. 验证步骤

完成上述配置后,请按照以下步骤验证问题是否解决:

  1. 检查虚拟机是否支持 AVX
    grep -o avx /proc/cpuinfo
    
    如果输出包含 avxavx2,说明虚拟机已正确支持 AVX。
  2. 启动 OceanBase 并检查日志
    obd cluster start <cluster_name>
    obd log <cluster_name>
    
    确保没有出现 cpu does not support avx 的错误信息。

4. 预防措施

  • 在部署 OceanBase 前,请确保虚拟机或物理服务器满足其最低硬件要求(包括 CPU、内存、磁盘等)。
  • 定期检查虚拟化平台的 CPU 配置,避免因配置变更导致兼容性问题。

5. 联系支持

如果上述方法仍无法解决问题,建议提供以下信息以便进一步排查:

  • 宿主机的 CPU 型号及特性(可通过 lscpu 查看)。
  • 虚拟机的 CPU 配置截图或详细信息。
  • OceanBase 的具体版本号及部署方式。

希望以上信息能帮助您解决问题!如果还有其他疑问,请随时提问。

具体信息可参考以下文档:

  1. enable_kv_group_commit
  2. tenant_cpu_variation_per_server
  3. enable_kv_ttl
  4. kv_group_commit_rw_mode
  5. 仲裁 Server 进程启动失败
  6. 10000 ~ 12000
  7. 备租户转为主租户
  8. 0001 ~ 3999

(小助手的答复已结束,如未能解决您的问题,请继续提问并等待其他同学的回复,谢谢!)

observer.zip (11.2 MB)

这个报错显示使用这个配置文件出现校验错误,可能是文件损坏或者磁盘异常。
可以对比下和observer.config.bin.history文件,如果内容变动不大,可以把原文件重命名,再修改history文件为原文件名称。
“etc/observer.config.bin”

[2025-05-21 10:29:22.959440] INFO  [SHARE] load_config (ob_config_manager.cpp:135) [1928][observer][T0][Y0-0000000000000000-0-0] [lt=14] Using config file(path="etc/observer.config.bin")
[2025-05-21 10:29:22.995916] WDIAG deserialize (ob_server_config.cpp:450) [1928][observer][T0][Y0-0000000000000000-0-0] [lt=37][errcode=-4007] object version mismatch(cls="ObServerConfig", ret=-4007, version=16382)
[2025-05-21 10:29:22.997494] WDIAG [COMMON] check_header_checksum (ob_record_header.cpp:56) [1928][observer][T0][Y0-0000000000000000-0-0] [lt=1566][errcode=-4103] record check checksum failed.(*this={magic:-257, header_length:1, version:584, header_checksum:0, timestamp:7825430413344, data_length:1, data_zlength:1497280, data_checksum:-2594073385365405696}, ret=-4103)
[2025-05-21 10:29:22.997520] ERROR issue_dba_error (ob_log.cpp:1875) [1928][observer][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-4103, file="ob_server_config.cpp", line_no=229, info="check header checksum failed")
[2025-05-21 10:29:22.998478] EDIAG [SHARE.CONFIG] deserialize_with_compat (ob_server_config.cpp:229) [1928][observer][T0][Y0-0000000000000000-0-0] [lt=956][errcode=-4103] check header checksum failed(ret=-4103) BACKTRACE:0x125fd785 0x50b6335 0x50b4e08 0x50b484f 0x50b478c 0x50b45bd 0xff2e12f 0xff17408 0xa7bcc39 0xa7b3165 0x7306344 0x7f2b8ab41d7a 0x5329d6e
[2025-05-21 10:29:23.018917] ERROR issue_dba_error (ob_log.cpp:1875) [1928][observer][T0][Y0-0000000000000000-0-0] [lt=20434][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-4103, file="ob_config_manager.cpp", line_no=150, info="Deserialize server config failed")
[2025-05-21 10:29:23.018952] EDIAG [SHARE] load_config (ob_config_manager.cpp:150) [1928][observer][T0][Y0-0000000000000000-0-0] [lt=34][errcode=-4103] Deserialize server config failed(path="etc/observer.config.bin", ret=-4103) BACKTRACE:0x125fd785 0x50b6335 0x51ad5ed 0x51ad0af 0x519a224 0x51acf53 0xff17c78 0xff17103 0xa7bcc39 0xa7b3165 0x7306344 0x7f2b8ab41d7a 0x5329d6e

还有这个目录名称 多了个 / 符号,也可以先重新设置下启动时的目录地址试试、
data_dir = //home/admin/observer/store

确实可以了,跟路径没关系,我大概知道为什么了,之前操作系统的 locale 不是en_US.UTF-8,所以导致这个文件在重写的时候失去了原本的编码,修订了以后就好了