【 使用环境 】测试环境
【 OB or 其他组件 】OB
【 使用版本 】4.3.1
【问题描述】使用obloader旁路导入，添加参数–direct --parallel=8 就导入失败，无法连接，不添加就可以
./obloader -h 10.27.192.223 -P 2883 -u root@edw16c64g -p ‘******’ --no-sys -D odata --cut --table ‘ods_sff_sffbusdocmain_yc’ --ctl-path /data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/ctrl/ods_sff_sffbusdocmain_yc.ctrl --log-path=/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/log/ods_sff_sffbusdocmain_yc -f /data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8.unl --column-splitter=x1b --thread 16 --rpc-port=2882
添加后报错信息如下
2024-08-23 10:34:07 [WARN] The Java version (1.8.0_192) you are currently using is not recommended. Recommended range: [1.8.0_300, 9.0.0)
2024-08-23 10:34:07 [INFO] Parsed args:
[–cut] true
[–ctl-path] /data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/ctrl/ods_sff_sffbusdocmain_yc.ctrl
[–file-path] /data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8.unl
[–no-sys] true
[–log-path] /data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/log/ods_sff_sffbusdocmain_yc
[–column-splitter] x1b
[–thread] 16
[–parallel] 8
[–host] 10.27.192.223
[–port] 2883
[–user] root@edw16c64g
[–password] ******
[–database] odata
[–sys-user] root
[–table] [ods_sff_sffbusdocmain_yc]
[–direct] true
[–rpc-port] 2882
2024-08-23 10:34:07 [INFO] Log files will be written to /data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/log/ods_sff_sffbusdocmain_yc
2024-08-23 10:34:07 [INFO] Trying to establish JDBC connection to root@edw16c64g…
2024-08-23 10:34:08 [INFO] Server Mode: OBMYSQL-4.3.1.0
2024-08-23 10:34:08 [INFO] Parse ctrl definition: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/ctrl/ods_sff_sffbusdocmain_yc.ctrl” success
2024-08-23 10:34:08 [INFO] Querying table column metadata, this might take a while…
2024-08-23 10:34:08 [INFO] Splitting data files into 64 MB logical chunks…
2024-08-23 10:34:09 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8.unl” has been splitted into 7717 copies
2024-08-23 10:34:09 [INFO] Split 1 data files to 7717 logical chunks success. Elapsed: 1.249 s
2024-08-23 10:34:09 [INFO] Bootstrap with Max Heap: 3 GB, Safe Heap: 3.09 GB
2024-08-23 10:34:09,598 main WARN No Root logger was configured, creating default ERROR-level Root logger with Console appender
2024-08-23 10:34:09 [INFO] Create 1024 slots for ring buffer finished. [0.0.0.0]
2024-08-23 10:34:09 [INFO] Start 16 database writer threads finished. [0.0.0.0]
2024-08-23 10:34:09 [INFO] Start 16 record file reader threads success
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-3.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-11.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-15.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-16.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-1.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-13.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-5.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-9.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-8.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-2.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-6.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-14.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-4.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-10.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-12.unl” has been parsed finished
2024-08-23 10:34:10 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8-7.unl” has been parsed finished
2024-08-23 10:34:11 [ERROR] Fatal error occurred while loading data from “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8.unl” into table odata.ods_sff_sffbusdocmain_yc. Reason: Write into table “ods_sff_sffbusdocmain_yc” failed. com.alipay.oceanbase.rpc.exception.ObTableServerConnectException: login failed after max 3 tries server [10.27.192.223:2882]
…全部都是这个
2024-08-23 10:34:31 [ERROR] Error: Write into table “ods_sff_sffbusdocmain_yc” failed. com.alipay.oceanbase.rpc.exception.ObTableServerConnectException: login failed after max 3 tries server [10.27.192.223:2882]
2024-08-23 10:34:31 [INFO] ---------- Finished Tasks: 7717 Running Tasks: 0 Progress: 100.00% ----------
2024-08-23 10:34:31 [INFO]
All Load Tasks Finished:
No.# | Type | Name | Count | Status
1 | TABLE | ods_sff_sffbusdocmain_yc | 48272 -> 0 | FAILURE
Total Count: 0 End Time: 2024-08-23 10:34:31
2024-08-23 10:34:31 [INFO] Load record finished. Total Elapsed: 23.73 s
2024-08-23 10:34:31 [ERROR] System exit 1
不添加 --direct --parallel=8 就可以正常导入
2024-08-23 10:39:37 [WARN] The Java version (1.8.0_192) you are currently using is not recommended. Recommended range: [1.8.0_300, 9.0.0)
2024-08-23 10:39:37 [INFO] Parsed args:
[–cut] true
[–ctl-path] /data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/ctrl/ods_sff_sffbusdocmain_yc.ctrl
[–file-path] /data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8.unl
[–no-sys] true
[–log-path] /data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/log/ods_sff_sffbusdocmain_yc
[–column-splitter] x1b
[–thread] 16
[–host] 10.27.192.223
[–port] 2883
[–user] root@edw16c64g
[–password] ******
[–database] odata
[–sys-user] root
[–table] [ods_sff_sffbusdocmain_yc]
[–rpc-port] 2882
2024-08-23 10:39:38 [INFO] Log files will be written to /data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/log/ods_sff_sffbusdocmain_yc
2024-08-23 10:39:38 [INFO] Trying to establish JDBC connection to root@edw16c64g…
2024-08-23 10:39:38 [INFO] Server Mode: OBMYSQL-4.3.1.0
2024-08-23 10:39:38 [INFO] Parse ctrl definition: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/ctrl/ods_sff_sffbusdocmain_yc.ctrl” success
2024-08-23 10:39:38 [INFO] Querying table column metadata, this might take a while…
2024-08-23 10:39:38 [INFO] Splitting data files into 64 MB logical chunks…
2024-08-23 10:39:39 [INFO] File: “/data1/APP/ODS/ob-loader-dumper-4.3.1-RELEASE/scrpits/data/ods_sff_sffbusdocmain_yc_utf8.unl” has been splitted into 7717 copies
2024-08-23 10:39:39 [INFO] Split 1 data files to 7717 logical chunks success. Elapsed: 1.256 s
2024-08-23 10:39:39 [INFO] Bootstrap with Max Heap: 3 GB, Safe Heap: 3.09 GB
2024-08-23 10:39:39 [INFO] Query table entry for table: “ods_sff_sffbusdocmain_yc” finished. Remain: 0
2024-08-23 10:39:39 [INFO] Query leader location of table: “ods_sff_sffbusdocmain_yc” finished. Remain: 0
2024-08-23 10:39:39 [INFO] Waiting to calculate leader for 7717 subfiles…
2024-08-23 10:39:40 [INFO] Waiting to refresh observer load status…
2024-08-23 10:39:40 [INFO] Create 1024 slots for ring buffer finished. [10.27.192.224:2881]
2024-08-23 10:39:40 [INFO] Start 16 database writer threads finished. [10.27.192.224:2881]
2024-08-23 10:39:40 [INFO] Start 16 record file reader threads success
2024-08-23 10:39:45 [INFO]
- Enqueue Performance Monitor:
Dimension \ Metric | Tps | Throughput | Buffer
1.sec.avg | 60790.2 Records/sec | 36.02 MB/sec | 1024 Slots
1.min.avg | 60505.2 Records/sec | 35.86 MB/sec | 1024 Slots
Total | 307230 Records | 182 MB | 1024 Slots
- Dequeue Performance Monitor:
Dimension \ Metric | Tps | Throughput | Buffer
1.sec.avg | 40784.56 Records/sec | 24.25 MB/sec | 1024 Slots
1.min.avg | 40082.0 Records/sec | 23.82 MB/sec | 1024 Slots
Total | 207466 Records | 123 MB | 1024 Slots
2024-08-23 10:39:49 [INFO] ---------- Finished Tasks: 0 Running Tasks: 7717 Progress: 0.06% ----------
