datax 导入oceanabse oracle模式 oceanbasev10writer 报错

yangmin · 2022 年10 月 15 日 10:48

使用环境】测试环境
【 OB or 其他组件】 DATAX
【使用版本】 githup上最新版本 DataX-datax_v202209.tar.gz
【问题描述】 oceanbasev10writer 报错： java.lang.NumberFormatException: multiple points
【复现路径】执行datax job 导入数据（ oceanbase oracle 模式写入问题）
【问题现象及影响】

源表和目标表表结构：
create table t1 (id VARCHAR2(30));

[root@pdmpcdp07 datax]# cat /root/datax/datax/job/oracle_dw_t1.json
{
“job”: {
“setting”: {
“speed”: {
“channel”: 1
}
},
“content”: [{
“reader”: {
“name”: “oraclereader”,
“parameter”: {
“username”: “edw_dba”,
“password”: “Hello0601”,
“connection”: [{
“querySql”: [
“select id from t1”
],
“jdbcUrl”: [
“jdbc:oracle:thin:@10.135.29.11:1521:edw2”
]
}]
}
},

  "writer": {
                "name": "oceanbasev10writer",
                "parameter": {
                    "obWriteMode": "insert",
                    "column": [
                        "ID"
                    ],
                    "connection": [
                        {
                            "jdbcUrl": "||_dsc_ob10_dsc_||yangmintest:tenant01||_dsc_ob10_dsc_||jdbc:oceanbase://10.249.240.1:2883/yangmin?",
                            "table": [
                                "t1"
                            ]
                        }
                    ],
                    "username": "yangmin",
                    "password":"yangmin"
                }
            }
        }
    ]
}

}

python /root/datax/datax/bin/datax.py /root/datax/datax/job/oracle_dw_t1.json

。。。。。。。。。。。。。。。

2022-10-15 10:24:34.411 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null
2022-10-15 10:24:34.412 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2022-10-15 10:24:34.413 [main] INFO JobContainer - DataX jobContainer starts job.
2022-10-15 10:24:34.414 [main] INFO JobContainer - Set jobId = 0
2022-10-15 10:24:34.712 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:oracle:thin:@10.135.29.11:1521:edw2.
2022-10-15 10:24:34.733 [job-0] INFO DBUtil - this is ob1_0 jdbc url.
2022-10-15 10:24:34.733 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=yangmintest:tenant01:yangmin :url=jdbc:oceanbase://10.249.240.1:2883/yangmin?
2022-10-15 10:24:34.967 [job-0] INFO DbUtils - value for query [SHOW VARIABLES LIKE ‘ob_compatibility_mode’] is [ORACLE]
2022-10-15 10:24:34.975 [job-0] INFO DBUtil - this is ob1_0 jdbc url.
2022-10-15 10:24:34.975 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=yangmintest:tenant01:yangmin :url=jdbc:oceanbase://10.249.240.1:2883/yangmin?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-10-15 10:24:35.017 [job-0] INFO OriginalConfPretreatmentUtil - table:[t1] all columns:[
ID
].
2022-10-15 10:24:35.018 [job-0] INFO DBUtil - this is ob1_0 jdbc url.
2022-10-15 10:24:35.018 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=yangmintest:tenant01:yangmin :url=jdbc:oceanbase://10.249.240.1:2883/yangmin?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-10-15 10:24:35.028 [job-0] INFO OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (ID) VALUES(?)
], which jdbcUrl like:[||dsc_ob10_dsc||yangmintest:tenant01||dsc_ob10_dsc||jdbc:oceanbase://10.249.240.1:2883/yangmin?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]
2022-10-15 10:24:35.029 [job-0] INFO JobContainer - jobContainer starts to do prepare …
2022-10-15 10:24:35.029 [job-0] INFO JobContainer - DataX Reader.Job [oraclereader] do prepare work .
2022-10-15 10:24:35.030 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] do prepare work .
2022-10-15 10:24:35.030 [job-0] INFO DBUtil - this is ob1_0 jdbc url.
2022-10-15 10:24:35.031 [job-0] INFO DBUtil - this is ob1_0 jdbc url. user=yangmintest:tenant01:yangmin :url=jdbc:oceanbase://10.249.240.1:2883/yangmin?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-10-15 10:24:35.047 [job-0] INFO DbUtils - value for query [show variables like ‘version’] is [3.2.3.0]
2022-10-15 10:24:35.047 [job-0] INFO JobContainer - jobContainer starts to do split …
2022-10-15 10:24:35.047 [job-0] INFO JobContainer - Job set Channel-Number to 1 channels.
2022-10-15 10:24:35.049 [job-0] INFO JobContainer - DataX Reader.Job [oraclereader] splits to [1] tasks.
2022-10-15 10:24:35.050 [job-0] INFO JobContainer - DataX Writer.Job [oceanbasev10writer] splits to [1] tasks.
2022-10-15 10:24:35.066 [job-0] INFO JobContainer - jobContainer starts to do schedule …
2022-10-15 10:24:35.068 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2022-10-15 10:24:35.070 [job-0] INFO JobContainer - Running by standalone Mode.
2022-10-15 10:24:35.076 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks.
2022-10-15 10:24:35.080 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2022-10-15 10:24:35.080 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2022-10-15 10:24:35.089 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2022-10-15 10:24:35.091 [0-0-0-writer] INFO OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-10-15 10:24:35.092 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select id from t1
] jdbcUrl:[jdbc:oracle:thin:@10.135.29.11:1521:edw2].
2022-10-15 10:24:35.092 [0-0-0-writer] INFO ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-10-15 10:24:35.103 [0-0-0-writer] INFO ConcurrentTableWriterTask - this is oracle compatible mode, change database to YANGMIN, table to T1
2022-10-15 10:24:35.103 [0-0-0-writer] INFO ConcurrentTableWriterTask - Disable partition calculation feature.
2022-10-15 10:24:35.114 [0-0-0-writer] INFO CommonRdbmsWriter$Task - write mode: insert
2022-10-15 10:24:35.115 [0-0-0-writer] INFO ConcurrentTableWriterTask - writeRecordSql :INSERT INTO T1 (ID) VALUES(?)
2022-10-15 10:24:35.119 [0-0-0-writer] ERROR WriterRunner - Writer Runner Received Exceptions:
java.lang.NumberFormatException: multiple points
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1914) ~[na:1.8.0_345]
at sun.misc.FloatingDecimal.parseFloat(FloatingDecimal.java:122) ~[na:1.8.0_345]
at java.lang.Float.parseFloat(Float.java:451) ~[na:1.8.0_345]
at java.lang.Float.valueOf(Float.java:416) ~[na:1.8.0_345]
at com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask.init(ConcurrentTableWriterTask.java:136) ~[oceanbasev10writer-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:222) ~[oceanbasev10writer-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:750) [na:1.8.0_345]
Exception in thread “taskGroup-0” com.alibaba.datax.common.exception.DataXException: Code:[Framework-13], Description:[DataX插件运行时出错, 具体原因请参看DataX运行结束时的错误诊断信息 .]. - java.lang.NumberFormatException: multiple points
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1914)
at sun.misc.FloatingDecimal.parseFloat(FloatingDecimal.java:122)
at java.lang.Float.parseFloat(Float.java:451)
at java.lang.Float.valueOf(Float.java:416)
at com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask.init(ConcurrentTableWriterTask.java:136)
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:222)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44)
at java.lang.Thread.run(Thread.java:750)

java.lang.NumberFormatException: multiple points
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1914)
at sun.misc.FloatingDecimal.parseFloat(FloatingDecimal.java:122)
at java.lang.Float.parseFloat(Float.java:451)
at java.lang.Float.valueOf(Float.java:416)
at com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask.init(ConcurrentTableWriterTask.java:136)
at com.alibaba.datax.plugin.writer.oceanbasev10writer.OceanBaseV10Writer$Task.init(OceanBaseV10Writer.java:222)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44)
at java.lang.Thread.run(Thread.java:750)

热海 · 2022 年10 月 15 日 13:13

稍等，oracel问题我们需要咨询一下商业团队

张雨齐 · 2022 年10 月 15 日 22:28

类型转换时,被转换的对象有多个小数点造成的。

应该是要检查数据吧，看看具体报错数据是不是不对

yangmin · 2022 年10 月 16 日 22:51

源表与目标表的表结构都是非常的简单，数据也仅有一条，即便我 id 换成 int 类型报错也是一样的。

源端 oracle 表结构和数据信息：
drop table t1;
create table t1 (id varchar2(30));
insert into t1 values(‘AAAA’);
commit;

目标： oceanbase oracle模式的DDL ：
show create table t1 ;
CREATE TABLE “T1” (
“ID” VARCHAR2(30)
) COMPRESS FOR ARCHIVE REPLICA_NUM = 3 BLOCK_SIZE = 16384 USE_BLOOM_FILTER = FALSE TABLET_SIZE = 134217728 PCTFREE = 0

yangmin · 2022 年10 月 16 日 22:57

ob 版本： 3.2.3 。

yangmin · 2022 年10 月 17 日 15:41

是需要我自己找商业团队么？还是有商业团队的技术支持来社区里处理？

热海 · 2022 年10 月 17 日 20:47

不好意思，我及时督促一下商业团队的同学过来协助解决

张雨齐 · 2022 年10 月 18 日 09:06

很有可能是需要datax的人处理吧。企业版的人也是负责OB，不负责dataX吧

杨守潮 · 2022 年10 月 18 日 11:45

好的，我们已经收到您的问题，正在为您查看，请稍等

杨守潮 · 2022 年10 月 18 日 14:30

您好，这边需要升级datax 版本解决问题，稍后我把tar 包发您

张雨齐 · 2022 年10 月 19 日 08:52

我想问这个版本是datax社区版，还是你们内部版本？

杨守潮 · 2022 年10 月 19 日 08:58

datax是开源的

张雨齐 · 2022 年10 月 19 日 09:03

明白，那其实还是楼主用错了开源版本datax。谢谢解答

yangmin · 2022 年10 月 19 日 15:58

已经测试过，升级后的datax 已经解决了问题。谢谢

wistwill · 2022 年10 月 20 日 21:52

请问这个问题最终是怎么解决的呢？最新的版本是要自己拉取master分支编译？

wistwill · 2022 年10 月 21 日 10:20

请问这个修复版本dataX是要从哪里获取呢？我从git上下载的是最新提供的V_202209版本的，但是还是有问题。可否发送一份修复版本的dataX呢，感谢

小糖人 · 2023 年8 月 17 日 16:31

您好可以发我一份最新的datax包吗？我也遇到了相同的问题

OB-sql · 2024 年3 月 11 日 21:27

我也遇到这个问题，怎么解决，新的版本在哪下，我的datax 3.0呀

OB-sql · 2024 年3 月 11 日 21:28

给我一个新的版本