oceanbase v4.2.2.1 + oblogproxy v2.0.1 + canal-for-ob 配置报错如何排查

OceanBase 技术问题
#1

【 使用环境 】测试环境
【 OB or 其他组件 】
【 使用版本 】
oceanbase v4.2.2.1
oblogproxy v2.0.1
canal-for-ob.deployer.tar.gz

【问题描述】
oblogproxy配置没有问题，可以生成binlog文件，并且位点在变化：
[root@oceanbase-04 data]# ls -lrt
total 2296304
-rw-rw-r-- 1 admin admin 0 May 13 17:44 purged.index
-rw-rw-r-- 1 admin admin 0 May 13 17:44 index.LOCK
-rw-rw-r-- 1 admin admin 524350969 May 15 12:40 mysql-bin.000001
-rw-rw-r-- 1 admin admin 524602732 May 17 03:20 mysql-bin.000002
-rw-rw-r-- 1 admin admin 524426706 May 18 12:00 mysql-bin.000003
-rw-rw-r-- 1 admin admin 524466636 May 20 08:11 mysql-bin.000004
-rw-rw-r-- 1 admin admin 123942809 May 20 15:12 mysql-bin.000005
-rw-rw-r-- 1 admin admin 738 May 20 15:12 mysql-bin.index

配置canal-for-ob后，启动正常，但日志报错：
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘com.oceanbase.clogproxy.client.config.ObReaderConfig#773c1f41’ defined in class path resource [spring/ob-default-instance.xml]: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanExpressionException: Expression parsing failed; nested exception is org.springframework.expression.ParseException: Expression [#{}] @0: No expression defined within delimiter ‘#{}’ at character 0
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:587)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:501)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:312)
… 53 common frames omitted
Caused by: org.springframework.beans.factory.BeanExpressionException: Expression parsing failed; nested exception is org.springframework.expression.ParseException: Expression [#{}] @0: No expression defined within delimiter ‘#{}’ at character 0
at org.springframework.context.expression.StandardBeanExpressionResolver.evaluate(StandardBeanExpressionResolver.java:164)
at org.springframework.beans.factory.support.AbstractBeanFactory.evaluateBeanDefinitionString(AbstractBeanFactory.java:1452)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.doEvaluate(BeanDefinitionValueResolver.java:266)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.evaluate(BeanDefinitionValueResolver.java:223)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:191)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1605)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1357)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:578)
… 55 common frames omitted
Caused by: org.springframework.expression.ParseException: Expression [#{}] @0: No expression defined within delimiter ‘#{}’ at character 0

instance.properties.txt (996 字节)
canal.properties.txt (6.6 KB)

#3

看起来是 extraConfigs 为空时读取出错了，在修复版本发布之前，您可以先随便设置一个 cdc 属性进去 OceanBase分布式数据库-海量数据 笔笔算数

#4

set extraConfigs for libobcdc, format {‘key1’: ‘value1’, ‘key2’: ‘value2’}

canal.instance.oceanbase.obcdc.extraConfigs={‘store_service_path’: ‘./storage’}

是这段配置信息吗？能给个示例吗？把注释去掉，重启canal，还是报错。

1 个赞
#5

2024-05-20 16:34:32.495 [Thread-33] ERROR c.a.o.c.p.inbound.oceanbase.logproxy.LogProxyConnection - OceanBase LogProxyClient listener error :
com.oceanbase.clogproxy.client.exception.LogProxyClientException: Unsupported protocol version: 19968
at com.oceanbase.clogproxy.client.connection.ClientHandler.checkHeader(ClientHandler.java:256)
at com.oceanbase.clogproxy.client.connection.ClientHandler.handleHeader(ClientHandler.java:174)
at com.oceanbase.clogproxy.client.connection.ClientHandler.channelRead(ClientHandler.java:141)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:266)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:1018)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:402)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:307)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
at java.lang.Thread.run(Thread.java:750)

#6

你好，去掉注释后没有报错了，但是canal拉不到数据。binlog是有新增的

#7

一开始没反应过来，您那边是用的 oblogproxy 的 binlog 模式吗？如果是的话，canal 需要用 mysql 的配置方式，把 OceanBase 完全当作 mysql 来用，ip 和 端口就用 obproxy jdbc 连接使用的 ip 和端口。

#8

我们用的是 oblogproxy 的 binlog 模式，如果OceanBase 完全当作 mysql 来用，是不是就不使用canal-for-ob的包了，使用mysql版本的canal就可以了？

#9

对的，可以直接用原版的 canal。

#10

用原版的也有报错：command : ‘show master status’ has an error! pls check. you need (at least one of) the SUPER,REPLICATION CLIENT privilege(s) for this operation

#11

直接用客户端连接 obproxy 执行 show master status 会有问题吗？需要注意用的账号要跟 canal 里用的一样。

#12

[root@oceanbase-04 ~]# mysql -h 172.28.18.13 -P 2883 -u canal@sys#ob_fat -pIZu_vB_9FpS3_6k6
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 1255261
Server version: 5.6.25 OceanBase_CE 4.2.2.0 (r100000192024011915-fac02c6690de9ff9f9f96c0bdf86ffe39ae0d7e7) (Built Jan 19 2024 15:14:05)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

MySQL [(none)]> show grants for canal@’%’;
±----------------------------------------------------------------------+
| Grants for canal@% |
±----------------------------------------------------------------------+
| GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON . TO ‘canal’ |
±----------------------------------------------------------------------+
1 row in set (0.01 sec)

MySQL [(none)]> show master status;
Empty set (0.24 sec)

MySQL [(none)]>

show master status命令没有任何返回。

#13

看起来 binlog service 没配对，正常情况下执行 show master status 应该会返回 binlog 相关的信息。我拉相关的同学看一下，稍后回复你。

#14

执行 show master status 需要在业务租户去执行，这里的业务租户对应 create binlog 语句中时创建的租户，不是 sys 租户中执行。

#15

[root@oceanbase-04 data]# mysql -h 172.28.18.13 -P 2883 -uroot@t_fat_01#ob_fat -p************
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 4187018
Server version: 5.6.25 OceanBase_CE 4.2.2.0 (r100000192024011915-fac02c6690de9ff9f9f96c0bdf86ffe39ae0d7e7) (Built Jan 19 2024 15:14:05)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

MySQL [(none)]> show master status ;
±-----------------±---------±-------------±-----------------±-------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
±-----------------±---------±-------------±-----------------±-------------------------------------------+
| mysql-bin.000001 | 239838 | | | 29060ce1-d08c-11ee-bc6f-52540007faf9:1-399 |
±-----------------±---------±-------------±-----------------±-------------------------------------------+
1 row in set (0.24 sec)

ok，业务租户有了，我们再试一下。

#16

总结：用原生的canal，没有使用ob版的canal，用业务租户的root用户，配置instance.properties文件，可以成功。谢谢两位@自凡和@川粉

#18

instance.properties文件是按照原生canal的方式配置吗