备份日志异常终止

OceanBase
,
#1

【 使用环境 】生产环境 or 测试环境
【 OB or 其他组件 】
【 使用版本 】
【问题描述】清晰明确描述问题
【复现路径】问题出现前后相关操作
【问题现象及影响】
系统发起备份alter system archivelog 后， 查询CDB_OB_BACKUP_ARCHIVELOG，发现状态时interrupted。 请问这种情况怎么排查错误呢？

谢谢
【附件】

#3

select log_archive_status,count(1) from __all_virtual_pg_backup_log_archive_status group by log_archive_status;
select svr_ip, svr_port, log_archive_status from __all_virtual_pg_backup_log_archive_status group by svr_ip, svr_port, log_archive_status;
select svr_ip, svr_port from __all_virtual_clog_stat where (table_id,partition_idx) in (select table_id,partition_id from __all_virtual_pg_backup_log_archive_status order by log_archive_cur_ts limit 3000) and role=“LEADER”;

#4

select log_archive_status,count(1) from __all_virtual_pg_backup_log_archive_status group by log_archive_status;
±-------------------±---------+
| log_archive_status | count(1) |
±-------------------±---------+
| 1 | 124 |
±-------------------±---------+

MySQL [oceanbase]> select svr_ip, svr_port, log_archive_status from __all_virtual_pg_backup_log_archive_status group by svr_ip, svr_port, log_archive_status;
±-------------±---------±-------------------+
| svr_ip | svr_port | log_archive_status |
±-------------±---------±-------------------+
| 192.168.56.2 | 2882 | 1 |
| 192.168.56.4 | 2882 | 1 |
±-------------±---------±-------------------+

#5

[2022-10-11 15:11:15.697447] ERROR [RS] do_schedule_ (ob_log_archive_scheduler.cpp:1187) [25868][212][YB42C0A83804-0005EAB8169EB2FA] [lt=29] [dc=0] [LOG_ARCHIVE] log archive status is interrupted, need manual process(sys_info={status:{tenant_id:1, copy_id:0, start_ts:1665469954085465, checkpoint_ts:0, status:5, incarnation:1, round:3, status_str:“INTERRUPTED”, is_mark_deleted:false, is_mount_file_created:true, compatible:1, backup_piece_id:0, start_piece_id:0}, backup_dest:“file:///home/backup”}) BACKTRACE:0x9a98e9e 0x986d141 0x22a7774 0x22a725b 0x22a6fc1 0x38b192c 0x70caba5 0x70c987b 0x6750a72 0x9a2aabd 0x9a2a4ee 0x340b9af 0x2cabf02 0x9820da5 0x981f792 0x981c24f
[2022-10-11 15:12:05.710843] ERROR [RS] do_schedule_ (ob_log_archive_scheduler.cpp:1187) [25868][212][YB42C0A83804-0005EAB8169EB2FA] [lt=44] [dc=0] [LOG_ARCHIVE] log archive status is interrupted, need manual process(sys_info={status:{tenant_id:1, copy_id:0, start_ts:1665469954085465, checkpoint_ts:0, status:5, incarnation:1, round:3, status_str:“INTERRUPTED”, is_mark_deleted:false, is_mount_file_created:true, compatible:1, backup_piece_id:0, start_piece_id:0}, backup_dest:“file:///home/backup”}) BACKTRACE:0x9a98e9e 0x986d141 0x22a7774 0x22a725b 0x22a6fc1 0x38b192c 0x70caba5 0x70c987b 0x6750a72 0x9a2aabd 0x9a2a4ee 0x340b9af 0x2cabf02 0x9820da5 0x981f792 0x981c24f
[2022-10-11 15:12:55.762913] ERROR [RS] do_schedule_ (ob_log_archive_scheduler.cpp:1187) [25868][212][YB42C0A83804-0005EAB8169EB2FA] [lt=28] [dc=0] [LOG_ARCHIVE] log archive status is interrupted, need manual process(sys_info={status:{tenant_id:1, copy_id:0, start_ts:1665469954085465, checkpoint_ts:0, status:5, incarnation:1, round:3, status_str:“INTERRUPTED”, is_mark_deleted:false, is_mount_file_created:true, compatible:1, backup_piece_id:0, start_piece_id:0}, backup_dest:“file:///home/backup”}) BACKTRACE:0x9a98e9e 0x986d141 0x22a7774 0x22a725b 0x22a6fc1 0x38b192c 0x70caba5 0x70c987b 0x6750a72 0x9a2aabd 0x9a2a4ee 0x340b9af 0x2cabf02 0x9820da5 0x981f792 0x981c24f

如上是rootservice.log里面的

#6

image
image1889×593 89.2 KB

#7

[2022-10-11 16:23:56.319368] ERROR [RS] do_schedule_ (ob_log_archive_scheduler.cpp:1187) [25868][212][YB42C0A83804-0005EAB8169EB2FA] [lt=12] [dc=0] [LOG_ARCHIVE] log archive status is interrupted, need manual process(sys_info={status:{tenant_id:1, copy_id:0, start_ts:1665473964540752, checkpoint_ts:0, status:5, incarnation:1, round:4, status_str:“INTERRUPTED”, is_mark_deleted:false, is_mount_file_created:true, compatible:1, backup_piece_id:0, start_piece_id:0}, backup_dest:“file:///home/backup”}) BACKTRACE:0x9a98e9e 0x986d141 0x22a7774 0x22a725b 0x22a6fc1 0x38b192c 0x70caba5 0x70c987b 0x6750a72 0x9a2aabd 0x9a2a4ee 0x340b9af 0x2cabf02 0x9820da5 0x981f792 0x981c24f