oms同步数据时Store 增量拉取组件失败了

[AntTech_ZGW8AE]

oceanbase社区版：4.3.5.3
oms：最新版本

从mysql同步数据到ob，反向增量这一步没有启动起来：

查看日志，libobcdc里面有error字样，log请见：
lbobcdc.log (549.0 KB)

麻烦帮忙看一下是什么原因，谢谢！

[咖啡哥]

没遇到过。你OMS哪个版本，进容器看看store版本。
docker exec -it oms bash
rpm -qa|grep store

ls_err:-4233
ls_err=“OB_ERR_OUT_OF_LOWER_BOUND”,

ERROR 4233 (HY000) : smaller than container lower bound

• OceanBase 错误码：4233
• 错误原因：小于容器下边界。
• 解决方式：请联系技术支持人员协助排查。

[AntTech_ZGW8AE]

@咖啡哥

docker exec -it oms bash
rpm -qa|grep store

版本是这个，今天上午刚装的
oboms-xlog-store-ce-5.7.14-72025071810.alios7.x86_64

在哪里可以联系技术？

[淇铭]

errcode=-4016clog被清理了
oms是哪个版本

[咖啡哥]

应该是你oms版本太低 了，当前版本的store不支持4.3.5.3
你这store版本看上去很老

[AntTech_ZGW8AE]

我是在这里下载的：
https://www.oceanbase.com/softwarecenter

不是新版本吗?

[咖啡哥]

看文档应该没问题。社区版的store版本可能跟企业版不一致。我以为很低呢
那应该不是版本的问题

[AntTech_ZGW8AE]

请问还有其他办法分析吗

[周雄]

WITH palf_log_stat AS (
SELECT
tenant_id,
MAX(begin_scn) AS palf_available_start_scn,
MIN(end_scn) AS palf_available_latest_scn,
SCN_TO_TIMESTAMP(MAX(begin_scn)) AS palf_available_start_scn_display,
SCN_TO_TIMESTAMP(MIN(end_scn)) AS palf_available_latest_scn_display
FROM GV$OB_LOG_STAT
WHERE tenant_id & 0x01 = 0 or tenant_id = 1
GROUP BY tenant_id
),
archivelog_stat AS (
SELECT
a.tenant_id AS tenant_id,
MIN(b.start_scn) AS archive_start_scn,
a.checkpoint_scn AS archive_latest_scn,
a.checkpoint_scn_display AS archive_available_latest_scn_display
FROM CDB_OB_ARCHIVELOG a
LEFT JOIN CDB_OB_ARCHIVELOG_PIECE_FILES b
ON a.tenant_id = b.tenant_id AND a.round_id = b.round_id
AND b.file_status != ‘DELETED’ AND a.STATUS = ‘DOING’
GROUP BY a.tenant_id
)
SELECT
pls.tenant_id,
pls.palf_available_start_scn,
pls.palf_available_latest_scn,
pls.palf_available_start_scn_display AS palf_available_start_scn_display,
pls.palf_available_latest_scn_display AS palf_available_latest_scn_display,
als.archive_start_scn AS archive_available_start_scn,
als.archive_latest_scn AS archive_available_latest_scn,
CASE WHEN als.archive_start_scn IS NOT NULL THEN SCN_TO_TIMESTAMP(als.archive_start_scn) ELSE NULL END AS archive_available_start_scn_dispalay,
als.archive_available_latest_scn_display
FROM palf_log_stat pls
LEFT JOIN archivelog_stat als ON pls.tenant_id = als.tenant_id
GROUP BY pls.tenant_id, pls.palf_available_start_scn;
ob系统租户oceanbase库执行一下，查一下位点

[AntTech_ZGW8AE]

@周雄

WITH palf_log_stat AS (
SELECT
tenant_id,
MAX(begin_scn) AS palf_available_start_scn,
MIN(end_scn) AS palf_available_latest_scn,
SCN_TO_TIMESTAMP(MAX(begin_scn)) AS palf_available_start_scn_display,
SCN_TO_TIMESTAMP(MIN(end_scn)) AS palf_available_latest_scn_display
FROM GV$OB_LOG_STAT
WHERE tenant_id & 0x01 = 0 or tenant_id = 1
GROUP BY tenant_id
),
archivelog_stat AS (
SELECT
a.tenant_id AS tenant_id,
MIN(b.start_scn) AS archive_start_scn,
a.checkpoint_scn AS archive_latest_scn,
a.checkpoint_scn_display AS archive_available_latest_scn_display
FROM CDB_OB_ARCHIVELOG a
LEFT JOIN CDB_OB_ARCHIVELOG_PIECE_FILES b
ON a.tenant_id = b.tenant_id AND a.round_id = b.round_id
AND b.file_status != ‘DELETED’ AND a.STATUS = ‘DOING’
GROUP BY a.tenant_id
)
SELECT
pls.tenant_id,
pls.palf_available_start_scn,
pls.palf_available_latest_scn,
pls.palf_available_start_scn_display AS palf_available_start_scn_display,
pls.palf_available_latest_scn_display AS palf_available_latest_scn_display,
als.archive_start_scn AS archive_available_start_scn,
als.archive_latest_scn AS archive_available_latest_scn,
CASE WHEN als.archive_start_scn IS NOT NULL THEN SCN_TO_TIMESTAMP(als.archive_start_scn) ELSE NULL END AS archive_available_start_scn_dispalay,
als.archive_available_latest_scn_display
FROM palf_log_stat pls
LEFT JOIN archivelog_stat als ON pls.tenant_id = als.tenant_id
GROUP BY pls.tenant_id, pls.palf_available_start_scn;

[AntTech_ZGW8AE]

是不是这个版本的oms和ob的4.3.5.3版本不匹配导致的，我记得前两天装的版本是4.3.5.2，是正常的。

[周雄]

EDIAG [TLOG.FETCHER] get_result (ob_log_start_lsn_locator.cpp:928) [769][][T0][Y7D26AC11AC58-0000000000100001-0-0] [lt=21][errcode=-4016] request start lsn from all server fail(tls_id={tenant_id:1002, ls_id:{id:1}}, start_tstamp_ns=1755130445380227000, svr_cnt=3, svr_list_consumed=3, result_svr_list_idx=-1, svr_list=[{svr:“172.17.172.84:2882”, rpc_executed:true, rpc_err:0,
这个报错是clog被清理了，反向增量的位点比 palf_available_start_scn_display 这个小，无法启动

还有目的端ob是不是没有开启归档

[周雄]

oms4210应该是支持这个版本的，你可以在oms容器内执行一下 ll /home/ds/lib64/reader/ob-ce-4.3-reader 看一下cdc版本是不是4.3.5.3

[AntTech_ZGW8AE]

1"这个报错是clog被清理了，反向增量的位点比 palf_available_start_scn_display 这个小，无法启动"
请问这个怎么解决

2 ob没有开启归档，但是前两天没有开，是可以的。

[AntTech_ZGW8AE]

[root@iZuf68men57hodmi6zgyg7Z ~]# ll /home/ds/lib64/reader/ob-ce-4.3-reader
total 1115104
-rwxr-xr-x 1 ds ds 460680 Nov 7 2024 libmariadb.so.3
-rwxr-xr-x 1 ds ds 496024 Jul 18 10:06 libob2store.so
lrwxrwxrwx 1 ds ds 19 Jul 22 16:22 libobcdc.so.4 → libobcdc.so.4.3.5.3
-rwxr-xr-x 1 ds ds 1140898224 Jul 17 19:05 libobcdc.so.4.3.5.3

[周雄]

这个store启动报错是在正向切换步骤报错的吗，正向切换截个图看一下

[AntTech_ZGW8AE]

最新的log
lbobcdc.log (548.1 KB)

[周雄]

clog没有了，而且归档也没开启，这个没法搞了。把ob归档打开或者clog磁盘扩大一点，重新建链路弄吧

[AntTech_ZGW8AE]

“或者clog磁盘扩大一点”
请问这是在哪里设置的？是安装oms的时候，还是创建迁移的时候？

[AntTech_ZGW8AE]

找到设置的地方了