ocp执行集群备份报错

【 使用环境 】生产环境
【 OB or 其他组件 】ocp
【 使用版本 】4.0
【问题描述】
ocp执行集群备份报错cannot find backup file


错误日志
2022-12-08 15:30:26.297 WARN 42 — [pool-manual-subtask-executor15,51f76897fcdf4d2c,acd22950117d] c.a.o.c.t.engine.runner.RunnerFactory : Execute task failed, subtask=SubtaskInstanceOverview{id=170226, name=Wait data backup task successful, state=FAILED, operation=EXECUTE, className=com.oceanbase.ocp.backup.internal.task.schedule.WaitDataBackupSuccessTask, seriesId=2, startTime=2022-12-08T15:30:05.740+08:00, endTime=null}, failedMessage=Data backup task failed.

56

57

com.alipay.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=BACKUP_DATA_BACKUP_TASK_FAILED, args=

58

at com.oceanbase.ocp.backup.internal.task.schedule.WaitDataBackupSuccessTask.lambda$checkDataBackupTaskResult$3(WaitDataBackupSuccessTask.java:142) ~[ocp-backup-service-4.0.0-20220930.jar!/:4.0.0-20220930]

59

at java.util.ArrayList.forEach(ArrayList.java:1259) ~[na:1.8.0_312]

60

at com.oceanbase.ocp.backup.internal.task.schedule.WaitDataBackupSuccessTask.checkDataBackupTaskResult(WaitDataBackupSuccessTask.java:132) ~[ocp-backup-service-4.0.0-20220930.jar!/:4.0.0-20220930]

61

at com.oceanbase.ocp.backup.internal.task.schedule.WaitDataBackupSuccessTask.run(WaitDataBackupSuccessTask.java:201) ~[ocp-backup-service-4.0.0-20220930.jar!/:4.0.0-20220930]

62

at com.alipay.ocp.core.task.engine.runner.JavaSubtaskRunner.execute(JavaSubtaskRunner.java:60) ~[ocp-core-4.0.0-20220930.jar!/:4.0.0-20220930]

63

at com.alipay.ocp.core.task.engine.runner.JavaSubtaskRunner.doRun(JavaSubtaskRunner.java:32) ~[ocp-core-4.0.0-20220930.jar!/:4.0.0-20220930]

64

at com.alipay.ocp.core.task.engine.runner.JavaSubtaskRunner.run(JavaSubtaskRunner.java:26) ~[ocp-core-4.0.0-20220930.jar!/:4.0.0-20220930]

65

at com.alipay.ocp.core.task.engine.runner.RunnerFactory.doRun(RunnerFactory.java:111) ~[ocp-core-4.0.0-20220930.jar!/:4.0.0-20220930]

66

at com.alipay.ocp.core.task.engine.runner.RunnerFactory.redirectOutputIfNotSysSchedule(RunnerFactory.java:183) ~[ocp-core-4.0.0-20220930.jar!/:4.0.0-20220930]

67

at com.alipay.ocp.core.task.engine.runner.Ru

68

nnerFactory.run(RunnerFactory.java:101) ~[ocp-core-4.0.0-20220930.jar!/:4.0.0-20220930]

69

at com.alipay.ocp.core.task.engine.coordinator.worker.subtask.ReadySubtaskWorker.lambda$null$1(ReadySubtaskWorker.java:117) ~[ocp-core-4.0.0-20220930.jar!/:4.0.0-20220930]

70

at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_312]

71

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_312]

72

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_312]

73

at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_312]

74

75

76

Set state for subtask: 170226, operation:EXECUTE, state: FAILED


observer日志
observer(3).7z (4.1 MB)

看下__all_virtual_backup_task表里哪些备份任务失败了


没看见信息

我也是同样的问题,OCP创建集群备份任务失败

麻烦捞一下rootservice的日志?

可以吧ocp 的前面的日志发一下吗

rootservice-20221209061607098.log (683.0 KB)
rootservice-20221209072248245.log (9.0 MB)

Wait data backup task successful.log (34.4 KB)

麻烦值班老师,帮看下,日志上面有人提供了,还需要什么吗?

[2022-12-09 06:15:54.933729] WARN [STORAGE] get_file_length (ob_storage_file.cpp:162) [12140][BackupMgr][T0][YB42C0A81C39-0005EF0B5F5C52DC-0-0] [lt=21] file not exist(ret=-9011, path="/data/obcluster/1/t
enant_incarnation_1/1004/data/backup_set_1_full/infos/meta_info/ls_attr_info_turn_1", errno=2, errno=“No such file or directory”)

看到rs日志中有这个报错信息,是否在每台observer上都挂在nfs了,nfs的挂载目录是/data吗,有没有删除过nfs中的文件呢

1 个赞

没删除过目录,是必须挂载nfs的目录吗?我是手动创建的目录,配置的权限

是的,要nfs