obdumper使用控制文件报错

bin/obdumper -h xxx -P 2883 -u root -t test -c prod  -p --sys-password  -D obcp4   --csv   -f 'dump2/q5.csv'  --table region  --ctl-path "ext/view.ctrl"
#  cat ext/view.ctrl
lang=java 
( 
[R_NAME] "lower(R_NAME)", 
[R_COMMENT] "upper(R_COMMENT)"  
); 

mysql> select * from region limit 10;
+-------------+-------------+---------------------------------------------------------------------------------------------------------------------+
| R_REGIONKEY | R_NAME      | R_COMMENT                                                                                                           |
+-------------+-------------+---------------------------------------------------------------------------------------------------------------------+
|           0 | AFRICA      | lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to  |
|           1 | AMERICA     | hs use ironic, even requests. s                                                                                     |
|           2 | ASIA        | ges. thinly even pinto beans ca                                                                                     |
|           3 | EUROPE      | ly final courts cajole furiously final excuse                                                                       |
|           4 | MIDDLE EAST | uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl        |
+-------------+-------------+---------------------------------------------------------------------------------------------------------------------+
5 rows in set (0.00 sec)
mysql> show create table region \G
*************************** 1. row ***************************
       Table: region
Create Table: CREATE TABLE `region` (
  `R_REGIONKEY` int(11) NOT NULL,
  `R_NAME` text NOT NULL,
  `R_COMMENT` text DEFAULT NULL,
  PRIMARY KEY (`R_REGIONKEY`)
) DEFAULT CHARSET = utf8mb4 ROW_FORMAT = DYNAMIC COMPRESSION = 'zstd_1.3.8' REPLICA_NUM = 3 BLOCK_SIZE = 16384 USE_BLOOM_FILTER = FALSE TABLET_SIZE = 134217728 PCTFREE = 0
1 row in set (0.00 sec)

报错如下:


2025-07-24 11:52:20 [WARN] The Java version (1.8.0_162) you are currently using is not recommended. Recommended range: [1.8.0_300, 9.0.0)
2025-07-24 11:52:21 [INFO] Log files will be written to dump2/logs
2025-07-24 11:52:21 [INFO] Trying to establish JDBC connection to `root@test#prod`...
2025-07-24 11:52:21 [INFO] Server Mode: OBMYSQL-4.2.1.8
2025-07-24 11:52:22 [INFO] Parse ctrl definition: "/data/ob-loader-dumper/ext/view.ctrl" success
2025-07-24 11:52:22 [INFO] Manifest "dump2/.MANIFEST.bin" has been saved
2025-07-24 11:52:22 [INFO] Querying table column metadata, this might take a while...
2025-07-24 11:52:22 [INFO] Filtering out empty tables...
2025-07-24 11:52:22 [INFO] Parallel write is set to false.
2025-07-24 11:52:22 [INFO] A total of 1 TABLE need to be dumped.
2025-07-24 11:52:23 [INFO] Query table entry for table "region" success. Elapsed: 1.183 s
2025-07-24 11:52:23 [INFO] Generating sub-tasks for table "region" by partitions and primary/unique keys...
2025-07-24 11:52:23 [ERROR] Split rows for non-partitioned table(with primary key): "region" failed. Reason: IllegalArgumentException: The column type is null. (`obcp4`.`region`.R_REGIONKEY)
2025-07-24 11:52:23 [ERROR] Failed to dump. java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: The column type is null. (`obcp4`.`region`.R_REGIONKEY)
        at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_162]
        at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_162]
        at com.oceanbase.tools.loaddump.dumper.AbstractDumper.doDumpAsync(AbstractDumper.java:300) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.dumper.AbstractDumper.dumpRecordAsync(AbstractDumper.java:148) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.client.DumpClient.dumpRecord(DumpClient.java:114) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.cmd.Obdumper.run(Obdumper.java:303) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.cmd.Obdumper.main(Obdumper.java:255) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
Caused by: java.lang.IllegalArgumentException: The column type is null. (`obcp4`.`region`.R_REGIONKEY)
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:445) ~[guava-32.0.1-jre.jar:?]
        at com.oceanbase.tools.loaddump.common.model.TableInfo.getColumnType(TableInfo.java:851) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.common.model.TableInfo.getColumnTypeName(TableInfo.java:866) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.dumper.task.generator.RecordDumpTaskGenerator.lambda$queryBounds$2(RecordDumpTaskGenerator.java:176) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_162]
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_162]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_162]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_162]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_162]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_162]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_162]
        at com.oceanbase.tools.loaddump.dumper.task.generator.RecordDumpTaskGenerator.queryBounds(RecordDumpTaskGenerator.java:177) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.dumper.task.generator.RecordDumpTaskGenerator.splitTableRows(RecordDumpTaskGenerator.java:136) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.dumper.task.generator.RecordDumpTaskGenerator.doGenerateDumpTasks(RecordDumpTaskGenerator.java:81) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.dumper.task.generator.AbstractSqlParserBasedRecordDumpTaskGenerator.generateDumpTasks(AbstractSqlParserBasedRecordDumpTaskGenerator.java:117) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.dumper.task.generator.RecordDumpTaskGenerator.generateDumpTasks(RecordDumpTaskGenerator.java:58) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at com.oceanbase.tools.loaddump.dumper.AbstractDumper.lambda$doDumpAsync$1(AbstractDumper.java:282) ~[ob-loader-dumper-4.3.3.1-RELEASE.jar:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_162]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_162]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_162]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_162]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_162]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_162]

2025-07-24 11:52:23 [ERROR] Dump failed! Error: The column type is null. (`obcp4`.`region`.R_REGIONKEY)
2025-07-24 11:52:23 [ERROR] System exit 1

看着像是数据有问题 报错主键列不能为空
image


mysql> show create table region \G
*************************** 1. row ***************************
       Table: region
Create Table: CREATE TABLE `region` (
  `R_REGIONKEY` int(11) NOT NULL,
  `R_NAME` text NOT NULL,
  `R_COMMENT` text DEFAULT NULL,
  PRIMARY KEY (`R_REGIONKEY`)
) DEFAULT CHARSET = utf8mb4 ROW_FORMAT = DYNAMIC COMPRESSION = 'zstd_1.3.8' REPLICA_NUM = 3 BLOCK_SIZE = 16384 USE_BLOOM_FILTER = FALSE TABLET_SIZE = 134217728 PCTFREE = 0
1 row in set (0.00 sec)

mysql> 
mysql> select * from region limit 10;
+-------------+-------------+---------------------------------------------------------------------------------------------------------------------+
| R_REGIONKEY | R_NAME      | R_COMMENT                                                                                                           |
+-------------+-------------+---------------------------------------------------------------------------------------------------------------------+
|           0 | AFRICA      | lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to  |
|           1 | AMERICA     | hs use ironic, even requests. s                                                                                     |
|           2 | ASIA        | ges. thinly even pinto beans ca                                                                                     |
|           3 | EUROPE      | ly final courts cajole furiously final excuse                                                                       |
|           4 | MIDDLE EAST | uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl        |
+-------------+-------------+---------------------------------------------------------------------------------------------------------------------+
5 rows in set (0.02 sec)

数据只有5条,如上

这个问题的根因是已知的会在 435迭代进行修复,主要原因是控制文件的规则将主键过滤了:
PK的类型被排除后,tableInfo的Columntypename中不会包含PK,导致queryBound无法获取。

1 个赞
# cat ext/view.ctrl
lang=java 
( 
[R_REGIONKEY] "ascii(R_REGIONKEY)",
[R_NAME] "lower(R_NAME)", 
[R_COMMENT] "upper(R_COMMENT)"  
); 

改成这样就正常了

1 个赞