datax 进行本地化演练环境linux环境单机版部署并执行从Mysql向oceanbase数据迁移时，当配置单个库进行迁移时，可正常同步，但当配置多个库并行同步时，则datax系统报错提示不支持，请协助定位一下问题，谢谢。
具体json配置如下：
{
“job”: {
“setting”: {
“speed”: {
“channel”: 4
},
“errorLimit”: {
“record”: 0,
“percentage”: 0.1
}
},
“content”: [
{
“reader”: {
“name”: “mysqlreader”,
“parameter”: {
“username”: “root”,
“password”: “whhszbd”,
“column”: [
“*”
],
“connection”: [
{
“table”: [
“test”
],
“jdbcUrl”: [“jdbc:mysql://10.30.120.195:3306/world?useUnicode=true&characterEncoding=utf8&useSSL=false&allowPublicKeyRetrieval=true”]
},
{
“table”: [
“testmysqlsync”
],
“jdbcUrl”: [“jdbc:mysql://10.30.120.195:3306/mysql?useUnicode=true&characterEncoding=utf8&useSSL=false&allowPublicKeyRetrieval=true”]
}
]
}
},
"writer": {
"name": "oceanbasev10writer",
"parameter": {
"obWriteMode": "insert",
"column": [
"*"
],
"preSql": [
"delete from test.test where user_id = 1"
],
"connection": [
{
"jdbcUrl":"jdbc:oceanbase://127.0.0.1:2881/test?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true",
"table": [
"test"
]
},
{
"jdbcUrl":"jdbc:oceanbase://127.0.0.1:2881/mysql?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true",
"table": [
"testmysqlsync"
]
}
],
"username": "hayden",
"password": "whhszbd",
"writerThreadCount":10,
"batchSize": 1000,
"memstoreThreshold": "0.9"
}
}
}
]
}
}
datax报错信息如下：
2024-04-01 12:00:19.766 [main] INFO MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2024-04-01 12:00:19.775 [main] INFO MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id=“GMT+08:00”,offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2024-04-01 12:00:19.999 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2024-04-01 12:00:20.023 [main] INFO Engine - the machine info =>
osInfo: Linux amd64 3.10.0-1160.71.1.el7.x86_64
jvmInfo: Red Hat, Inc. 1.8 25.332-b09
cpu num: 1
totalPhysicalMemory: -0.00G
freePhysicalMemory: -0.00G
maxFileDescriptorCount: -1
currentOpenFileDescriptorCount: -1
GC Names [Copy, MarkSweepCompact]
MEMORY_NAME | allocation_size | init_size
Eden Space | 273.06MB | 273.06MB
Code Cache | 240.00MB | 2.44MB
Survivor Space | 34.13MB | 34.13MB
Compressed Class Space | 1,024.00MB | 0.00MB
Metaspace | -0.00MB | 0.00MB
Tenured Gen | 682.69MB | 682.69MB
2024-04-01 12:00:20.138 [main] INFO Engine -
{
“setting”:{
“speed”:{
“channel”:4
},
“errorLimit”:{
“record”:0,
“percentage”:0.1
}
},
“content”:[
{
“reader”:{
“name”:“mysqlreader”,
“parameter”:{
“username”:“root”,
“password”:“",
“column”:[
""
],
“connection”:[
{
“table”:[
“test”
],
“jdbcUrl”:[
“jdbc:mysql://10.30.120.195:3306/world?useUnicode=true&characterEncoding=utf8&useSSL=false&allowPublicKeyRetrieval=true”
]
},
{
“table”:[
“testmysqlsync”
],
“jdbcUrl”:[
“jdbc:mysql://10.30.120.195:3306/mysql?useUnicode=true&characterEncoding=utf8&useSSL=false&allowPublicKeyRetrieval=true”
]
}
]
}
},
“writer”:{
“name”:“oceanbasev10writer”,
“parameter”:{
“obWriteMode”:“insert”,
“column”:[
""
],
“preSql”:[
“delete from test.test where user_id = 1”
],
“connection”:[
{
“jdbcUrl”:“jdbc:oceanbase://127.0.0.1:2881/test?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true”,
“table”:[
“test”
]
},
{
“jdbcUrl”:“jdbc:oceanbase://127.0.0.1:2881/mysql?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true”,
“table”:[
“testmysqlsync”
]
}
],
“username”:“hayden”,
“password”:"**”,
“writerThreadCount”:10,
“batchSize”:1000,
“memstoreThreshold”:“0.9”
}
}
}
]
}
2024-04-01 12:00:20.388 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false
2024-04-01 12:00:20.389 [main] INFO JobContainer - DataX jobContainer starts job.
2024-04-01 12:00:20.407 [main] INFO JobContainer - Set jobId = 0
2024-04-01 12:00:23.063 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://10.30.120.195:3306/world?useUnicode=true&characterEncoding=utf8&useSSL=false&allowPublicKeyRetrieval=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2024-04-01 12:00:23.139 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://10.30.120.195:3306/mysql?useUnicode=true&characterEncoding=utf8&useSSL=false&allowPublicKeyRetrieval=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2024-04-01 12:00:23.146 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2024-04-01 12:00:24.279 [job-0] INFO DbUtils - value for query [SHOW VARIABLES LIKE ‘ob_compatibility_mode’] is [MYSQL]
2024-04-01 12:00:24.409 [job-0] INFO OriginalConfPretreatmentUtil - table:[test] all columns:[
user_id,username,sex,age,state
].
2024-04-01 12:00:24.414 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2024-04-01 12:00:24.421 [job-0] INFO OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (user_id,username,sex,age,state) VALUES(?,?,?,?,?)
], which jdbcUrl like:[jdbc:oceanbase://127.0.0.1:2881/test?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]
2024-04-01 12:00:24.440 [job-0] INFO JobContainer - jobContainer starts to do prepare …
2024-04-01 12:00:24.448 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2024-04-01 12:00:24.455 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] do prepare work .
2024-04-01 12:00:24.584 [job-0] INFO OceanBaseV10Writer$Job - Begin to execute preSqls:[delete from test.test where user_id = 1]. context info:jdbc:oceanbase://127.0.0.1:2881/test?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2024-04-01 12:00:24.752 [job-0] INFO OceanBaseV10Writer$Job - Begin to execute preSqls:[delete from test.test where user_id = 1]. context info:jdbc:oceanbase://127.0.0.1:2881/mysql?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2024-04-01 12:00:24.776 [job-0] INFO JobContainer - jobContainer starts to do split …
2024-04-01 12:00:24.780 [job-0] INFO JobContainer - Job set Channel-Number to 4 channels.
2024-04-01 12:00:24.968 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] splits to [2] tasks.
2024-04-01 12:00:24.977 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] splits to [2] tasks.
2024-04-01 12:00:25.470 [job-0] INFO JobContainer - jobContainer starts to do schedule …
2024-04-01 12:00:25.502 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2024-04-01 12:00:25.512 [job-0] INFO JobContainer - Running by standalone Mode.
2024-04-01 12:00:25.560 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [2] channels for [2] tasks.
2024-04-01 12:00:25.604 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2024-04-01 12:00:25.610 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2024-04-01 12:00:25.739 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2024-04-01 12:00:25.770 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] attemptCount[1] is started
2024-04-01 12:00:25.776 [0-0-0-writer] ERROR WriterRunner - Writer Runner Received Exceptions:
java.lang.RuntimeException: writing to multi-tables is not supported.
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:221) ~[oceanbasev10writer-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:750) [na:1.8.0_332]
2024-04-01 12:00:25.786 [0-0-1-writer] ERROR WriterRunner - Writer Runner Received Exceptions:
java.lang.RuntimeException: writing to multi-tables is not supported.
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:221) ~[oceanbasev10writer-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:750) [na:1.8.0_332]
2024-04-01 12:00:25.809 [0-0-1-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from testmysqlsync
] jdbcUrl:[jdbc:mysql://10.30.120.195:3306/mysql?useUnicode=true&characterEncoding=utf8&useSSL=false&allowPublicKeyRetrieval=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2024-04-01 12:00:25.809 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from test
] jdbcUrl:[jdbc:mysql://10.30.120.195:3306/world?useUnicode=true&characterEncoding=utf8&useSSL=false&allowPublicKeyRetrieval=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2024-04-01 12:00:26.171 [0-0-1-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from testmysqlsync
] jdbcUrl:[jdbc:mysql://10.30.120.195:3306/mysql?useUnicode=true&characterEncoding=utf8&useSSL=false&allowPublicKeyRetrieval=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2024-04-01 12:00:26.239 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from test
] jdbcUrl:[jdbc:mysql://10.30.120.195:3306/world?useUnicode=true&characterEncoding=utf8&useSSL=false&allowPublicKeyRetrieval=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2024-04-01 12:00:35.669 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
2024-04-01 12:00:35.680 [job-0] ERROR JobContainer - 运行scheduler 模式[standalone]出错.
2024-04-01 12:00:35.703 [job-0] ERROR JobContainer - Exception when job run
com.alibaba.datax.common.exception.DataXException: Code:[Framework-13], Description:[DataX插件运行时出错, 具体原因请参看DataX运行结束时的错误诊断信息 .]. - java.lang.RuntimeException: writing to multi-tables is not supported.
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:221)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44)
at java.lang.Thread.run(Thread.java:750)
-
java.lang.RuntimeException: writing to multi-tables is not supported.
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:221)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44)
at java.lang.Thread.run(Thread.java:750)at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:48) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.scheduler.processinner.ProcessInnerScheduler.dealFailedStat(ProcessInnerScheduler.java:39) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.scheduler.AbstractScheduler.schedule(AbstractScheduler.java:99) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.schedule(JobContainer.java:535) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:119) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.start(Engine.java:86) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.entry(Engine.java:168) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.main(Engine.java:201) [datax-core-0.0.1-SNAPSHOT.jar:na]
java.lang.RuntimeException: writing to multi-tables is not supported.
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:221) ~[na:na]
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:750) ~[na:1.8.0_332]
2024-04-01 12:00:35.796 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
2024-04-01 12:00:35.804 [job-0] ERROR Engine -
经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[Framework-13], Description:[DataX插件运行时出错, 具体原因请参看DataX运行结束时的错误诊断信息 .]. - java.lang.RuntimeException: writing to multi-tables is not supported.
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:221)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44)
at java.lang.Thread.run(Thread.java:750)
-
java.lang.RuntimeException: writing to multi-tables is not supported.
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:221)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44)
at java.lang.Thread.run(Thread.java:750)at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:48)
at com.alibaba.datax.core.job.scheduler.processinner.ProcessInnerScheduler.dealFailedStat(ProcessInnerScheduler.java:39)
at com.alibaba.datax.core.job.scheduler.AbstractScheduler.schedule(AbstractScheduler.java:99)
at com.alibaba.datax.core.job.JobContainer.schedule(JobContainer.java:535)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:119)
at com.alibaba.datax.core.Engine.start(Engine.java:86)
at com.alibaba.datax.core.Engine.entry(Engine.java:168)
at com.alibaba.datax.core.Engine.main(Engine.java:201)
Caused by: java.lang.RuntimeException: writing to multi-tables is not supported.
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:221)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44)
at java.lang.Thread.run(Thread.java:750)