datax全量迁移从mariadb到oceanbase失败。Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]

【 使用环境 】
测试环境
【 OB or 其他组件 】
DataX
【 使用版本 】
mariadb版本:

MariaDB [(none)]> select version();
+---------------------+
| version()           |
+---------------------+
| 10.3.10-MariaDB-log |
+---------------------+

OB版本

obclient [oceanbase]> select version();
+--------------------+
| version()          |
+--------------------+
| 3.1.4-OceanBase CE |
+--------------------+

【问题描述】
执行python bin/datax.py job/customer1.jso时报错

2023-07-26 10:52:25.323 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第1次重试.本次重试计划等待[1000]ms,实际等待[1000]ms, 异常Msg:[Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).].  -  具体错误信息为:java.lang.ClassNotFoundException: com.alipay.oceanbase.jdbc.Driver]
2023-07-26 10:52:25.324 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2023-07-26 10:52:25.325 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obce-3zones:tenant1:root :url=jdbc:oceanbase://172.118.81.156:2883/nova?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true
2023-07-26 10:52:27.327 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第2次重试.本次重试计划等待[2000]ms,实际等待[2000]ms, 异常Msg:[Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).].  -  具体错误信息为:java.lang.ClassNotFoundException: com.alipay.oceanbase.jdbc.Driver]

json文件如下:

{
    "job": {
        "setting": {
            "speed": {
                "channel": 4 
            },
            "errorLimit": {
                "record": 0,
                "percentage": 0.1
            }
        },
        "content": [
            {
                "reader": {
                    "name": "mysqlreader",
                    "parameter": {
                        "username": "root",
                        "password": "cloudadmin*******",
                        "column": [
                            "*"
                        ],
                        "connection": [
                            {
                                "table": [
                                    "config"
                                ],
                                "jdbcUrl": ["jdbc:mysql://10.192.45.36:3306/nova?useUnicode=true&characterEncoding=utf8"]
                            }
                        ]
                    }
                },

                "writer": {
                    "name": "oceanbasev10writer",
                    "parameter": {
                        "obWriteMode": "insert",
                        "column": [
                            "*"
                        ],
                        "preSql": [
                            "truncate table config"
                        ],
                        "connection": [
                            {
                                "jdbcUrl": "||_dsc_ob10_dsc_||obce-3zones:tenant1||_dsc_ob10_dsc_||jdbc:oceanbase://172.118.81.156:2883/nova?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true",
                                "table": [
                                    "config"
                                ]
                            }
                        ],
                        "username": "root",
                        "password": "cloudadmin",
                        "writerThreadCount":10,
                        "batchSize": 1000,
                        "memstoreThreshold": "0.9"
                    }
                }
            }
        ]
    }
}

使用json文件mysqlreader中的用户和密码登录mariadb

-bash-4.2$ mysql -h10.192.45.36 -uroot -p nova
Enter password: 
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 3305
Server version: 10.3.10-MariaDB-log MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [nova]> 

使用json文件oceanbasev10writer中的用户和密码登录OB

-bash-4.2$ obclient -h172.118.81.156 -uobce-3zones:tenant1:root -P2883 -pcloudadmin -c -A nova
Welcome to the OceanBase.  Commands end with ; or \g.
Your OceanBase connection id is 113
Server version: OceanBase 3.1.4 (r103000102023020719-16544a206f00dd3ceb4ca3011a625fbb24568154) (Built Feb  7 2023 19:32:02)

Copyright (c) 2000, 2018, OceanBase and/or its affiliates. All rights reserved.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

obclient [nova]> 

经测试mariadb和OB的用户名和密码都是没问题的,是不是json文件中的哪里出问题了
【复现路径】
【问题现象及影响】

【附件】

1 个赞

看报错信息是找不到driver的类

2 个赞

用的 datax 是什么版本

1 个赞

datax是从github上下载的。下载地址
解压后关于mysql和oceanbase的文件有

-bash-4.2$ find plugin/ -type f |egrep 'mysqlreader|oceanbasev10writer'
plugin/reader/mysqlreader/plugin_job_template.json
plugin/reader/mysqlreader/plugin.json
plugin/reader/mysqlreader/libs/commons-collections-3.0.jar
plugin/reader/mysqlreader/libs/druid-1.0.15.jar
plugin/reader/mysqlreader/libs/commons-lang3-3.3.2.jar
plugin/reader/mysqlreader/libs/logback-core-1.0.13.jar
plugin/reader/mysqlreader/libs/commons-io-2.4.jar
plugin/reader/mysqlreader/libs/mysql-connector-java-5.1.47.jar
plugin/reader/mysqlreader/libs/datax-common-0.0.1-SNAPSHOT.jar
plugin/reader/mysqlreader/libs/guava-r05.jar
plugin/reader/mysqlreader/libs/plugin-rdbms-util-0.0.1-SNAPSHOT.jar
plugin/reader/mysqlreader/libs/hamcrest-core-1.3.jar
plugin/reader/mysqlreader/libs/logback-classic-1.0.13.jar
plugin/reader/mysqlreader/libs/commons-math3-3.1.1.jar
plugin/reader/mysqlreader/libs/slf4j-api-1.7.10.jar
plugin/reader/mysqlreader/libs/fastjson2-2.0.23.jar
plugin/reader/mysqlreader/mysqlreader-0.0.1-SNAPSHOT.jar
plugin/writer/oceanbasev10writer/plugin.json
plugin/writer/oceanbasev10writer/libs/oceanbase-connector-java-3.2.0.jar
plugin/writer/oceanbasev10writer/libs/log4j-1.2.16.jar
plugin/writer/oceanbasev10writer/libs/commons-collections-3.0.jar
plugin/writer/oceanbasev10writer/libs/druid-1.0.15.jar
plugin/writer/oceanbasev10writer/libs/commons-lang3-3.3.2.jar
plugin/writer/oceanbasev10writer/libs/logback-core-1.0.13.jar
plugin/writer/oceanbasev10writer/libs/commons-io-2.4.jar
plugin/writer/oceanbasev10writer/libs/datax-common-0.0.1-SNAPSHOT.jar
plugin/writer/oceanbasev10writer/libs/guava-r05.jar
plugin/writer/oceanbasev10writer/libs/plugin-rdbms-util-0.0.1-SNAPSHOT.jar
plugin/writer/oceanbasev10writer/libs/hamcrest-core-1.3.jar
plugin/writer/oceanbasev10writer/libs/json-20160810.jar
plugin/writer/oceanbasev10writer/libs/logback-classic-1.0.13.jar
plugin/writer/oceanbasev10writer/libs/commons-math3-3.1.1.jar
plugin/writer/oceanbasev10writer/libs/slf4j-api-1.7.10.jar
plugin/writer/oceanbasev10writer/libs/fastjson2-2.0.23.jar
plugin/writer/oceanbasev10writer/oceanbasev10writer-0.0.1-SNAPSHOT.jar

完整的任务日志可以发下吗?

1 个赞

刚刚又执行了一遍,日志如下
x_job_customer1_json-17_08_22.963.log (9.0 KB)

更新一下master的代码吧 之前应该是缺少依赖了

master代码如何更新? :pray:

用master分支重新打个包就可以了

1 个赞

请老师再指点一二,从git的master分支下载的DataX-master.zip代码后,需要怎么做?有没有文档或链接。

$ md5sum DataX-master.zip 
42e755f8d7e83e532ba4489fa0b73b77  DataX-master.zip

重新打下包就可以啦 mvn package assembly:assembly -Dmaven.test.skip=true -s settings.xml

1 个赞