【 使用环境 】生产环境
【 OB or 其他组件 】 OCP 管理集群备份
【 使用版本 】 V 4.3.5
【问题描述】 OCP 管理集群备份,备份minio S3存储介质失败
S3用户权限设置

OCP备份配置
备份错误显示
我也按照测试方式去测试了一下
./ob_admin test_io_device
-d’s3://obbackup-bucket/newsrec_obbackup/’
-s’host=http://minio-content.service.xxx.org:8380&access_id=xxxx&access_key=xxxxxx’
日志如下:
ob_admin_rs.log (4.0 KB)
ob_admin.log (94.3 KB)
【附件及日志】推荐使用OceanBase敏捷诊断工具obdiag收集诊断信息,详情参见链接(右键跳转查看):
【SOP系列 22 】——故障诊断第一步(自助诊断和诊断信息收集)
希望帮忙排查一下 。
3 个赞
旭辉
#4
从ob_admin.log看有两个报错,第2个报错麻烦先用aws的命令行工具测试下list接口兼容性
https://docs.aws.amazon.com/cli/latest/reference/s3/
报错1:
[2025-04-21 15:48:55.444944] WDIAG [LIB] ObPool (ob_pool.ipp:37) [785583][][T500][Y0-0000000000000000-0-0] [lt=18][errcode=-4016] obj size larger than block size(obj_size_=7960, block_size_=7936)
[2025-04-21 15:48:55.444963] WDIAG [LIB] ObPool (ob_pool.ipp:37) [785583][][T500][Y0-0000000000000000-0-0] [lt=19][errcode=-4016] obj size larger than block size(obj_size_=8536, block_size_=7936)
[2025-04-21 15:48:55.444969] WDIAG [LIB] ObPool (ob_pool.ipp:37) [785583][][T500][Y0-0000000000000000-0-0] [lt=5][errcode=-4016] obj size larger than block size(obj_size_=10192, block_size_=7936)
报错2:
[2025-04-21 15:48:55.449388] WDIAG Log (ob_storage_s3_base.cpp:68) [785583][][T500][Y0-0000000000000000-0-0] [lt=8][errcode=-9105] [S3] module=CurlHttpClient, msg=Curl returned error code 6 - Error
[2025-04-21 15:48:55.449445] WDIAG Log (ob_storage_s3_base.cpp:68) [785583][][T500][Y0-0000000000000000-0-0] [lt=13][errcode=-9105] [S3] module=AWSXmlClient, msg=HTTP response code: -1
Resolved remote host IP address:
Request ID:
Exception name:
Error message: curlCode: 6, Error
0 response headers:
[2025-04-21 15:48:55.449487] WDIAG Log (ob_storage_s3_base.cpp:68) [785583][][T500][Y0-0000000000000000-0-0] [lt=7][errcode=-9105] [S3] module=AWSClient, msg=If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
[2025-04-21 15:48:55.449497] WDIAG Log (ob_storage_s3_base.cpp:68) [785583][][T500][Y0-0000000000000000-0-0] [lt=5][errcode=-9105] [S3] module=AWSClient, msg=Request failed, now waiting 0 ms before attempting again.
[2025-04-21 15:48:55.450307] WDIAG Log (ob_storage_s3_base.cpp:68) [785583][][T500][Y0-0000000000000000-0-0] [lt=4][errcode=-9105] [S3] module=CurlHttpClient, msg=Curl returned error code 6 - Error
[2025-04-21 15:48:55.450351] WDIAG Log (ob_storage_s3_base.cpp:68) [785583][][T500][Y0-0000000000000000-0-0] [lt=10][errcode=-9105] [S3] module=AWSXmlClient, msg=HTTP response code: -1
Resolved remote host IP address:
Request ID:
Exception name:
Error message: curlCode: 6, Error
0 response headers:
[2025-04-21 15:48:55.450390] WDIAG Log (ob_storage_s3_base.cpp:68) [785583][][T500][Y0-0000000000000000-0-0] [lt=15][errcode=-9105] [S3] module=AWSClient, msg=If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
[2025-04-21 15:48:55.450422] WDIAG log_s3_status (ob_storage_s3_base.cpp:655) [785583][][T500][Y0-0000000000000000-0-0] [lt=5][errcode=-9105] S3 info(request_id="", code=-1, exception="", err_msg="curlCode: 6, Error")
[2025-04-21 15:48:55.450434] WDIAG do_list_ (ob_storage_s3_base.cpp:1049) [785583][][T500][Y0-0000000000000000-0-0] [lt=11][errcode=-9105] failed to list s3 objects(ret=-9105, bucket=obbackup-bucket, object=newsrec_obbackup/2025-04-21-15:48:55/, max_list_num=1000, delimiter=NULL)
[2025-04-21 15:48:55.450454] WDIAG list_files_ (ob_storage_s3_base.cpp:1517) [785583][][T500][Y0-0000000000000000-0-0] [lt=17][errcode=-9105] fail to list s3 objects(ret=-9105, uri=s3://obbackup-bucket/newsrec_obbackup/2025-04-21-15:48:55/)
[2025-04-21 15:48:55.450478] WDIAG [STORAGE] list_files (ob_storage.cpp:704) [785583][][T500][Y0-0000000000000000-0-0] [lt=8][errcode=-9105] fail to list files(ret=-9105, uri=s3://obbackup-bucket/newsrec_obbackup/2025-04-21-15:48:55)
[2025-04-21 15:48:55.450491] WDIAG inner_scan_dir_ (ob_object_device.cpp:633) [785583][][T500][Y0-0000000000000000-0-0] [lt=7][errcode=-9105] fail to do list/dir scan!(ret=-9105, is_dir_scan=false, dir_name=s3://obbackup-bucket/newsrec_obbackup/2025-04-21-15:48:55)
[2025-04-21 15:48:55.450502] WDIAG is_empty_directory (ob_backup_io_adapter.cpp:738) [785583][][T500][Y0-0000000000000000-0-0] [lt=10][errcode=-9105] fail to scan dir!(ret=-9105, uri=s3://obbackup-bucket/newsrec_obbackup/2025-04-21-15:48:55)
[2025-04-21 15:48:55.450515] WDIAG [STORAGE] test_backup_dest_connectivity_ (ob_admin_test_io_device_executor.cpp:128) [785583][][T500][Y0-0000000000000000-0-0] [lt=7][errcode=-9105] failed check backup dest is empty dir(ret=-9105, backup_path="s3://obbackup-bucket/newsrec_obbackup/2025-04-21-15:48:55")
[2025-04-21 15:48:55.450564] WDIAG [STORAGE] run_all_tests_ (ob_admin_test_io_device_executor.cpp:105) [785583][][T500][Y0-0000000000000000-0-0] [lt=7][errcode=-9105] test backup dest connectivity failed(ret=-9105, backup_path="s3://obbackup-bucket/newsrec_obbackup/2025-04-21-15:48:55")
[2025-04-21 15:48:55.450575] WDIAG [STORAGE] execute (ob_admin_test_io_device_executor.cpp:50) [785583][][T500][Y0-0000000000000000-0-0] [lt=7][errcode=-9105] failed to pass all tests(ret=-9105, backup_path="s3://obbackup-bucket/newsrec_obbackup/2025-04-21-15:48:55")
[2025-04-21 15:48:55.450584] WDIAG [COMMON] main (main.cpp:160) [785583][][T500][Y0-0000000000000000-0-0] [lt=7][errcode=-9105] Fail to executor cmd, (ret=-9105)
4 个赞
旭辉
#5
第2个报错也提示访问s3存储失败,解析远端IP地址失败,测试下这台机器访问这个s3存储的连通性
curl -v bucket-name.host
例如:
curl -v http://s3-436751-abc-dataa-all-test.s3.cn-north-1.amazonaws.com.cn
5 个赞
旭辉
#10
ob 4219 开始支持 S3 对象存储访问新增可选参数 addressing_model
支持 S3 路径配置可选参数 addressing_model,可选参数值:virtual_hosted_style/path_style。S3 路径配置addressing_model=path_style 后,支持以 path style addressing model 访问对象存储;配置 addressing_model=virtual_hosted_style 或不配置可选参数 addressing_model 默认以 virtual hosted style 访问对象存储。
一般自建的s3仅支持path_style方式,
目前ocp还不支持,内部已经有排期了,如果需要这种方式,当前只能通过命令行的方式
SET LOG_ARCHIVE_DEST
https://www.oceanbase.com/docs/common-oceanbase-database-cn-1000000002016884
SET DATA_BACKUP_DEST
https://www.oceanbase.com/docs/common-oceanbase-database-cn-1000000002016885
1 个赞
旭辉
#11
结论:
自建s3仅支持path style方式,OCP预计4.3.6支持path style,目前不支持
绕过方式:配置直接使用IP访问minio