【 使用环境 】测试环境
【 OB or 其他组件 】oceanbase 3节点集群
【 使用版本 】OceanBase_CE 4.1.0.0 (r100000202023040520-0765e69043c31bf86e83b5d618db0530cf31b707) (Built Apr 5 2023 20:26:14)
【问题描述】
之前服务器因断电重启后，oceanbase用obd部署的3节点集群无法启动，执行
obd cluster start obtest --wop
启动后报错
之前为了测试多租户，建了10个2G的小租户
【附件及日志】
rootservice.log日志看似乎是租户出现问题
[2024-01-25 09:47:13.295819] WDIAG [RS] refresh_tenant_info (ob_tenant_info_loader.cpp:215) [41800][T1008_TenantInf][T1008][Y0-0000000000000000-0-0] [lt=12][errcode=-8002] failed to load tenant info(ret=-8002, ret=“OB_SERVER_IS_STOPPING”, tenant_id=1008)
[2024-01-25 09:47:13.295828] WDIAG [RS] run2 (ob_tenant_info_loader.cpp:130) [41800][T1008_TenantInf][T1008][Y0-0000000000000000-0-0] [lt=9][errcode=-8002] failed to update tenant info(ret=-8002, ret=“OB_SERVER_IS_STOPPING”, tenant_id=1008, sql_proxy_=0xe9afa08)
[2024-01-25 09:47:13.296724] WDIAG [RS] refresh_tenant_info (ob_tenant_info_loader.cpp:215) [41322][T1006_TenantInf][T1006][Y0-0000000000000000-0-0] [lt=9][errcode=-8002] failed to load tenant info(ret=-8002, ret=“OB_SERVER_IS_STOPPING”, tenant_id=1006)
[2024-01-25 09:47:13.296732] WDIAG [RS] run2 (ob_tenant_info_loader.cpp:130) [41322][T1006_TenantInf][T1006][Y0-0000000000000000-0-0] [lt=7][errcode=-8002] failed to update tenant info(ret=-8002, ret=“OB_SERVER_IS_STOPPING”, tenant_id=1006, sql_proxy_=0xe9afa08)
[2024-01-25 09:47:13.302272] WDIAG [RS] get_tenant_info (ob_tenant_info_loader.cpp:249) [41449][T1006_STSWorker][T1006][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] tenant info is invalid, need wait(ret=-4076, ret=“OB_NEED_WAIT”, refresh_time_us_=-1, tenant_info_={tenant_id:0, tenant_role:{value:0}, switchover_status:{value:0}, switchover_epoch:-1, sync_scn:{val:0}, replayable_scn:{val:0}, standby_scn:{val:0}, recovery_until_scn:{val:0}, log_mode:{archive mode:“INVALID”}})
[2024-01-25 09:47:13.331686] INFO [RS] run2 (ob_tenant_recovery_reportor.cpp:133) [41321][T1006_TeRec][T1006][Y0-0000000000000000-0-0] [lt=7] tenant schema not ready(ret=0, ret=“OB_SUCCESS”, meta_tenant_id=1005)
[2024-01-25 09:47:13.331706] WDIAG [RS] run2 (ob_tenant_recovery_reportor.cpp:145) [41321][T1006_TeRec][T1006][Y0-0000000000000000-0-0] [lt=0][errcode=0] failed to submit_tenant_refresh_schema_task_(tmp_ret=-4076, tmp_ret=“OB_NEED_WAIT”)
[2024-01-25 09:47:13.331729] WDIAG [RS] update_replayable_point_ (ob_tenant_recovery_reportor.cpp:347) [41321][T1006_TeRec][T1006][Y0-0000000000000000-0-0] [lt=0][errcode=-4008] update_replayable_point_from_meta_ failed(ret=-4008, ret=“OB_ITER_END”)
[2024-01-25 09:47:13.331736] WDIAG [RS] run2 (ob_tenant_recovery_reportor.cpp:150) [41321][T1006_TeRec][T1006][Y0-0000000000000000-0-0] [lt=6][errcode=0] failed to update_replayable_point(tmp_ret=-4008, tmp_ret=“OB_ITER_END”)
执行 obd display-trace
启动报错trace.txt (49.1 KB)