通过OCP进行租户备份报错

[LShangFi]

在使用ocp进行集群其中一个租户的备份时，其中一个步骤抛出如下报错：

查看对应时间节点日志并无相关报错：

望解答

[LShangFi]

数据库为三节点集群

[try_again]

你好，完整的日志提供下

[LShangFi]

subtask_30278.log (20.9 KB)

[旭辉]

麻烦确认下ocp版本

业务集群OB的版本

show variables like '%version_comment%';

[LShangFi]

[旭辉]

status=FAILED, progress=null, errorMsg=(ROOTSERVICE)addr: XXX.XXX.X.31:2882, result: -9120(cannot find object on object storage)

使用的是NFS备份介质吧？

麻烦取下备份报错时段的observer.log，rootservice.log 或者重新发起一次备份操作 然后取下最新的observer.log，rootservice.log 上传下

取WITH_ROOTSERVER='yes’的就行

select * from dba_ob_servers where WITH_ROOTSERVER='yes';

[LShangFi]

还有个问题，通过sql设置某租户LOG_ARCHIVE_DEST为目录A并开启归档后，在ocp上启动该租户的备份，并设置备份目录B后，LOG_ARCHIVE_DEST好像也会自动被改为目录B？

[LShangFi]

集群是必须要使用NFS是吧？但是好像把备份目录设置成NFS的目录后，归档的目录也会自动被改为该目录？这对归档会有什么影响吗？

[旭辉]

是的，在ocp上配置的备份 备份目录和归档日志目录 上层目录是一样的，在ocp上配置备份时黑屏就不要配置归档目录了。

[旭辉]

配置备份时，备份和归档目录都必须是NFS或者OSS，COS，S3等存储介质并且所有observer都必须可以访问的到这些备份介质

[LShangFi]

已经配置了NFS，重新发起备份还是在第五步出错
observer.log等.zip (31.2 MB)

[LShangFi]

报错时间在2025-01-20 16:37:20左右

[小小的人儿]

解决了吗？一般ocp报错可以先重试一下

[LShangFi]

还没有，已经重试好多次了

[旭辉]

fail to do list/dir scan!(ret=-5331, dir_name=file:///data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog, is_dir_scan=true, is_marker_scan=false)

这里1001租户的归档日志怎么指到1002的归档目录了？
做了什么操作吗？

[2025-01-20 15:28:12.530381] INFO  [ARCHIVE] checkpoint_ (ob_tenant_archive_scheduler.cpp:692) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=36] schedule archive(round_info={key:{tenant_id:1002, dest_no:0}, incarnation:1, dest_id:1004, round_id:2, state:{status:"STOP"}, start_scn:{val:1737017933841924000, v:0}, checkpoint_scn:{val:1737103285230399003, v:0}, max_scn:{val:1737103391800645002, v:0}, compatible:{version:1}, base_piece_id:2, used_piece_id:2, piece_switch_interval:86400000000, frozen_input_bytes:651880464, frozen_output_bytes:651880464, active_input_bytes:0, active_output_bytes:0, deleted_input_bytes:0, deleted_output_bytes:0, path:"file:///data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog", comment:""})
[2025-01-20 15:28:12.535184] WDIAG [SHARE] list_directories (ob_storage_file.cpp:688) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=35][errcode=-5331] fail to operate dir entry(ret=-5331, entry.d_name=check_file)
[2025-01-20 15:28:12.535226] INFO  list_directories (ob_storage_file.cpp:700) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=37] list directories count(dir_path=/data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog/, ret=-5331)
[2025-01-20 15:28:12.535234] WDIAG [STORAGE] list_directories (ob_storage.cpp:1244) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=7][errcode=-5331] failed to list_files(ret=-5331, uri=file:///data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog, uri_buf="file:///data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog/")
[2025-01-20 15:28:12.535249] WDIAG inner_scan_dir_ (ob_object_device.cpp:773) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=13][errcode=-5331] fail to do list/dir scan!(ret=-5331, dir_name=file:///data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog, is_dir_scan=true, is_marker_scan=false)
[2025-01-20 15:28:12.538591] INFO  [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=8] get number(ret=0, service_type_=0, range=1, base_id=1737358092538468000, start_id=1737358092538468000, end_id=1737358092538468001)
[2025-01-20 15:28:12.542618] INFO  [SHARE] update_last_check_time (ob_backup_connectivity.cpp:985) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=31] update backup last check time in storage info(sql=UPDATE __all_backup_storage_info SET last_check_time = 1737358092537543 WHERE tenant_id = 1002 AND path = 'file:///data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog' AND endpoint = '', tenant_id=1002, backup_dest={root_path:"file:///data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog", storage_info:{endpoint:"", access_id:"", extension:"", type:"FILE", checksum_type:0, max_iops:0, max_bandwidth:0}})
[2025-01-20 15:28:12.542649] INFO  [SHARE] check_backup_dest_connectivity (ob_backup_connectivity.cpp:214) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=29] [BACKUP_DEST_CHECK] succeed to finish backup_dest connectivity check(tenant_id=1002, backup_dest={root_path:"file:///data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog", storage_info:{endpoint:"", access_id:"", extension:"", type:"FILE", checksum_type:0, max_iops:0, max_bandwidth:0}})
[2025-01-20 15:28:12.542839] WDIAG [SHARE] check_dest_validity (ob_backup_store.cpp:459) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=8][errcode=-9080] format file does not exist(ret=-9080, backup_dest={root_path:"file:///data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog", storage_info:{endpoint:"", access_id:"", extension:"", type:"FILE", checksum_type:0, max_iops:0, max_bandwidth:0}})
[2025-01-20 15:28:12.542862] WDIAG check_dest_validity (ob_backup_store.cpp:460) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=20][errcode=-9080] format file does not exist, try to set a new directory.
[2025-01-20 15:28:12.542869] WDIAG [ARCHIVE] check_archive_dest_validity_ (ob_tenant_archive_scheduler.cpp:511) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=5][errcode=-9080] fail to check archive dest validity(ret=-9080, tenant_id_=1002, dest_str=file:///data/oceanbase-ce/oceanbase/backup/szjg_cluster/1731558683/tenant_incarnation_1/1002/clog)
[2025-01-20 15:28:12.542883] WDIAG [ARCHIVE] enable_archive (ob_tenant_archive_scheduler.cpp:578) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=12][errcode=-9080] fail to check archive dest valivity(ret=-9080, tenant_id=1002, dest_no=0)
[2025-01-20 15:28:12.542895] WDIAG [RS] process_ (ob_archive_scheduler_service.cpp:336) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=12][errcode=-9080] failed to enable archive(ret=-9080, tenant_id=1002, dest_no=0, dest_state={state:0})
[2025-01-20 15:28:12.542907] WDIAG [RS] run2 (ob_archive_scheduler_service.cpp:91) [18136][T1001_ArchiveSv][T1001][YB42C0B4071F-00062BA60912C02F-0-0] [lt=10][errcode=-9080] failed to do process(tmp_ret=-9080)

[旭辉]

另外ocp上任务日志也麻烦下载发下

[LShangFi]

并没有做什么额外的操作

[LShangFi]

log_task_32670.zip (8.3 KB)

[LShangFi]

该租户ID确实是1002