【 使用环境 】测试环境
【 OB or 其他组件 】
【 使用版本 】seatunnel2.3.11、oceanbase版本忘记了
【问题描述】目前使用seatunnel2.3.11的版本已经能实现从mysql同步数据到dorsi,oceanbase同步至doris了。但是最近有需求需要从oceanbase的a表同步到oceanbase的b表,但是出现了一些问题,因为a、b两张表的字段并非完全一致,需要做字段映射。
以下是seatunnel的同步配置:
env {
parallelism = 1
job.mode = "BATCH"
}
source {
Jdbc {
driver = "com.oceanbase.jdbc.Driver"
url = "jdbc:oceanbase://127.0.0.1:2883/test?useUnicode=true&characterEncoding=UTF-8"
user = "myUser"
password = "myPassword"
compatible_mode = "mysql"
query = """
SELECT
tenant_code,
create_by
FROM test_322
"""
}
}
sink {
Jdbc {
url = "jdbc:oceanbase://127.0.0.1:2883/test?useUnicode=true&characterEncoding=UTF-8"
driver = "com.oceanbase.jdbc.Driver"
user = "myUser"
password = "myPassword"
compatible_mode = "mysql"
query = "INSERT INTO test_323(tenant_code, create_by) VALUES(?, ?)"
columns = ["tenant_code", "create_by"]
batch_size = 1000
batch_interval_ms = 1000
}
}
报错如下:
Caused by: com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException: Unknown column 'id' in 'field list'
[127.0.0.1:2882] [2025-07-04 14:52:49.542714] [YB420A4FBDC4-000627985B4C6A0F-0-0]
If column exists but type cannot be identified (example 'select ? `field1` from dual'). Use CAST function to solve this problem (example 'select CAST(? as integer) `field1` from dual')
明明id是自增的,在source也没有查出id,但是却报这个错误。
调试时也时常出现以下错误:
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[COMMON-08], ErrorDescription:[Sql operation failed, such as (execute,addBatch,close) etc...] - Couldn't read data - bad format, For input string: "DM-5"
另外我试过一个版本是在source的query语句上把select的字段都带上单引号’,就能同步到b表,条数是对的,但是除了自增id有值以外其他字段都是NULL的。
目前配置是去掉字段映射的,要是大佬有时间也请指点以下如何映射?比如:a表的a字段 赋值给 b表的b字段,类型是一致的,只是字段名不一致。seatunnel官网不太友好。谢谢大佬们。
请大佬有时间指点以下,困扰一天了。
【复现路径】seatunnel2.3.11,直接使用配置文件执行。
./bin/seatunnel.sh --config ./job/st-ob2ob.conf -m local
【附件及日志】推荐使用OceanBase敏捷诊断工具obdiag收集诊断信息,详情参见链接(右键跳转查看):
【备注】基于 LLM 和开源文档 RAG 的论坛小助手已开放测试,在发帖时输入 [@论坛小助手] 即可召唤小助手,欢迎试用!