ocp 发起备份报错 start log archive backup when not STOP is not supported

OceanBase 技术问题
,
#1

【 使用环境 】 测试环境
【 OB or 其他组件 】ocp ob
【 使用版本 】
3.1
【问题描述】清晰明确描述问题
备份的磁盘路径已经设置好，只是通过ocp发起备份的审核提示 error code [1235]; message [(conn=205207) start log archive backup when not STOP is not supported
不知是否是有其他参数需要设置？

手动发起archivelog命令就是会报这个错误

MySQL [oceanbase]> ALTER SYSTEM ARCHIVELOG;
ERROR 1235 (0A000): start log archive backup when not STOP is not supported

【复现路径】问题出现前后相关操作
【问题现象及影响】

【附件日志】
日志如下
2022-08-29 10:11:09.366 ERROR 142 — [pool-subtask-executor-thread-39,cc45bfa90d2b4414,f65ff83f569f] c.a.o.c.obsdk.connector.ConnectTemplate : [obsdk] execute failed, sql:[alter system archivelog], error message:[StatementCallback; SQL [alter system archivelog]; (conn=205207) start log archive backup when not STOP is not supported; nested exception is java.sql.SQLFeatureNotSupportedException: (conn=205207) start log archive backup when not STOP is not supported]

2022-08-29 10:11:09.368 ERROR 142 — [pool-subtask-executor-thread-39,cc45bfa90d2b4414,f65ff83f569f] c.o.o.b.i.o.PhysicalBackupObOpsService : SQL [alter system archivelog]; SQL state [0A000]; error code [1235]; message [(conn=205207) start log archive backup when not STOP is not supported]

2022-08-29 10:11:09.369 INFO 142 — [pool-subtask-executor-thread-39,cc45bfa90d2b4414,f65ff83f569f] c.a.o.c.m.t.model.SubtaskInstanceEntity : Set state for subtask: 1650312, current state: RUNNING, new state: FAILED
2022-08-29 10:11:09.371 WARN 142 — [pool-subtask-executor-thread-39,cc45bfa90d2b4414,f65ff83f569f] c.a.o.c.t.engine.runner.RunnerFactory : Execute task failed, subtask=SubtaskInstanceEntity{id=1650312, name=Start log backup if necessary, state=FAILED, operation=EXECUTE, className=com.oceanbase.ocp.backup.internal.task.schedule.StartLogBackupTask, seriesId=1, startTime=2022-08-29T10:11:09.265+08:00, endTime=2022-08-29T10:11:09.370+08:00}, failedMessage=Failed to start log backup. Error message: (conn=205207) start log archive backup when not STOP is not supported.

com.alipay.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=BACKUP_START_LOG_BACKUP_FAILED, args=(conn=205207) start log archive backup when not STOP is not supported

20

at com.oceanbase.ocp.backup.internal.operation.PhysicalBackupObOpsService.startLogBackup(PhysicalBackupObOpsService.java:419) ~[ocp-backup-service-3.2.3-20220426.jar!/:3.2.3-20220426]

at com.oceanbase.ocp.backup.internal.operation.PhysicalBackupObOpsService.startLogBackup(PhysicalBackupObOpsService.java:400)

执行全量备份失败
#3

是不是已经有备份在执行过程中。
查一下这个表

select * from __all_backup_log_archive_status_v2;
#4

之前备份过几次，都是报这个错误
MySQL [oceanbase]> select * from __all_backup_log_archive_status_v2 \G;
*************************** 1. row ***************************
gmt_create: 2022-06-30 20:01:11.448926
gmt_modified: 2022-08-24 18:58:53.598949
tenant_id: 1
incarnation: 1
log_archive_round: 1
min_first_time: 2022-08-24 17:48:11.296532
max_next_time: 2022-08-24 18:54:34.794266
input_bytes: 0
output_bytes: 0
deleted_input_bytes: 0
deleted_output_bytes: 0
pg_count: 0
status: INTERRUPTED
is_mount_file_created: 0
compatible: 1
start_piece_id: 0
backup_piece_id: 0
backup_dest: file:///data/log1/backup
*************************** 2. row ***************************
gmt_create: 2022-08-24 17:48:11.356529
gmt_modified: 2022-08-24 18:59:43.604829
tenant_id: 1001
incarnation: 1
log_archive_round: 1
min_first_time: 2022-08-24 17:48:11.296532
max_next_time: 2022-08-24 18:54:34.794266
input_bytes: 0
output_bytes: 0
deleted_input_bytes: 0
deleted_output_bytes: 0
pg_count: 0
status: INTERRUPTED
is_mount_file_created: 0
compatible: 1
start_piece_id: 0
backup_piece_id: 0
backup_dest: file:///data/log1/backup
2 rows in set (0.01 sec)

#5
  1. 先从内部表
    select * from __all_rootservice_event_history where module like ‘%archive%’ order by gmt_create desc limit 30;
    查看value1=INTERRUPTED那一行的rs_srv_ip字段确认是哪个rs ip在处理日志归档
  2. 到对应的机器上帮忙查一下grep -rws “log archive status is interrupted” rootservice.log* 看看日志归档是否断流了？
#6

1 查询结果如下

1661744590868
16617445908683342×803 66.2 KB

2 在interrupt节点上 没有过滤出报错信息
[root@obtest1 log]# grep -rws “log archive status is interrupted” rootservice.log*
[root@obtest1 log]#

#7

麻烦再看下这个视图：select * from CDB_OB_BACKUP_ARCHIVELOG

#8

MySQL [oceanbase]> select * from CDB_OB_BACKUP_ARCHIVELOG
→ ;
±------------±------------------±----------±------------±---------------±----------------±---------------------------±---------------------------±------------±-------------±------------------±--------------------±---------------------+
| INCARNATION | LOG_ARCHIVE_ROUND | TENANT_ID | STATUS | START_PIECE_ID | BACKUP_PIECE_ID | MIN_FIRST_TIME | MAX_NEXT_TIME | INPUT_BYTES | OUTPUT_BYTES | COMPRESSION_RATIO | INPUT_BYTES_DISPLAY | OUTPUT_BYTES_DISPLAY |
±------------±------------------±----------±------------±---------------±----------------±---------------------------±---------------------------±------------±-------------±------------------±--------------------±---------------------+
| 1 | 1 | 1 | INTERRUPTED | 0 | 0 | 2022-08-24 17:48:11.296532 | 2022-08-24 18:54:34.794266 | 0 | 0 | NULL | 0.00MB | 0.00MB |
| 1 | 1 | 1001 | INTERRUPTED | 0 | 0 | 2022-08-24 17:48:11.296532 | 2022-08-24 18:54:34.794266 | 0 | 0 | NULL | 0.00MB | 0.00MB |
±------------±------------------±----------±------------±---------------±----------------±---------------------------±---------------------------±------------±-------------±------------------±--------------------±---------------------+

#9

其他节点有没有相关的打印呢？
目前看是因为8月24号出现了日志断流，导致后续无法发起新的日志备份，排查下8月24号左右的日志？如果rootservice.log 上没有，查一下observer.log* ？

#10

大佬，我也碰到这个问题，查询结果跟帖主差不多：

image
image1577×225 86.6 KB

请问这个情况后续要怎么处理呢？

#11

这样的 ，需要在ocp的容器里创建对应的备份目录就好了。也就是说ocp的备份平台是认为在容器里的，而不是ocp所在的宿主机的目录