TPCH ACI测试实战

准备

  1. 下载tpch相关脚本(见附件),解压到个人指定目录(<you_script_path>)
  2. 准备一台 16c64G 的机器(理论上机器资源也可以更小,不过没有实际测试过)

1. 集群部署

https://open.oceanbase.com/docs/observer-cn/V3.1.3/10000000000096602

2. 租户创建

obd cluster tenant create <deploy_name> -n test

这里注意下,deploy_name需要和集群部署时填的deploy name一致,否则会有报错,下同

3. TPCH测试

参照:https://open.oceanbase.com/docs/observer-cn/V3.1.3/10000000000096610

3.1-安装依赖

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://mirrors.aliyun.com/oceanbase/OceanBase.repo
sudo yum install obtpch
sudo ln -s /usr/tpc-h-tools/tpc-h-tools/ /usr/local/ 

3.2-运行测试

# --ddl-path里面需要填“准备”部分下载的资源包中的解压出来对应的位置
obd test tpch <deploy_name> --tenant=test -s 1 --ddl-path=<you_script_path>/oceanbase_tpch/benchmark_scripts/DDL/create_tpch_mysql_table_part.ddl --remote-tbl-dir=/tmp/tpch1g 

4. TPCH ACID验证

TPCH官方发布的FDR中有关于ACID验证的描述,有兴趣的同学可以了解一下:https://www.tpc.org/results/fdr/tpch/oceanbase~tpch~30000~oceanbase_v32~fdr~2021-05-19~v01.pdf

官网提供的ACID验证的脚本有部分硬编码和环境相关的变量,需要做一些调整之后才可以执行,具体如下:

4.1-修改config.sh

cd <you_script_path>/benchmark_scripts/KIT/script vim config.sh 

需要将TPCHDIR修改成<you_script_path>/oceanbase_tpch/benchmark_scripts/KIT/

4.2-修改数据库连接信息(可选,如果使用默认创建的tenant,本步骤可不执行)

cd <you_script_path>/oceanbase_tpch/benchmark_scripts/KIT/ACID/scripts
vim run_acid.sh

需要修改DBNAME、TPC_USER、TPC_TENANT

4.3-重新编译acid的binary(可选,已经有编译好的版本,如果用不了需要重新编译)

# 安装依赖 
wget http://dev.mysql.com/get/mysql57-community-release-el7-8.noarch.rpm
rpm -ivh mysql57-community-release-el7-8.noarch.rpm
rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022
sudo yum -y install mysql
# 编译
cd <you_script_path>/oceanbase_tpch/benchmark_scripts/KIT/ACID/scripts/transaction_isolation
gcc -g -O0 -o acidtransaction acidtransactionmain.c querystreamexec.h querystreamexec.c -Wl,-rpath=/usr/lib64 -Wl,-rpath=/usr/lib64.mysql:/u01/obclient/lib -lpthread -lmysqlclient -I /u01/obclient/include -L/u01/obclient/lib
cd <you_script_path>/oceanbase_tpch/benchmark_scripts/KIT/ACID/scripts/transaction_atomicity/
gcc -g -O0 -o acidtransaction acidtransactionmain.c querystreamexec.h querystreamexec.c -Wl,-rpath=/usr/lib64 -Wl,-rpath=/usr/lib64.mysql:/u01/obclient/lib -lpthread -lmysqlclient -I /u01/obclient/include -L/u01/obclient/lib
cd <you_script_path>/oceanbase_tpch/benchmark_scripts/KIT//script/query_streams/
gcc -g -O0 -o querystreamex querystreammain.c querystreamexec.c -Wl,-rpath=/usr/lib64 -Wl,-rpath=/usr/lib64.mysql:/u01/obclient/lib -lpthread -lmysqlclient -I /u01/obclient/include -L/u01/obclient/lib -L./

4.4-修改默认配置项和系统参数;

# tpch acid 依赖prepare statement默认打开
obclient -h<your_ip> -P<your_port> -uroot@sys -A -Dtest -e "alter system set _ob_enable_prepared_statement = True;"
# tpch acid 部分语句查询结果返回较慢,调大默认超时时间
obclient -h<your_ip> -P<your_port> -uroot@test -A -Dtest -e "set @@global.ob_query_timeout = 100000000;"

4.5-运行ACID测试

cd <you_script_path>/oceanbase_tpch/benchmark_scripts/KIT/ACID/scripts
# your_ip your_port需要填部署时指定的server ip、port
sh run_acid.sh <your_ip> <your_port> ./ 1 10 ./ ./

4.6-最终结果

最终结果会生成在

<you_script_path>/oceanbase_tpch/benchmark_scripts/KIT/ACID/scripts/ACID

FAQ

line 69: \[: Q1: integer expression expected
line 74: \[: Q1: integer expression expected 

如果机器比较小,跑isolation6的时候,可能会遇到上述报错;该报错主要原因是Q1查询没能在足够时间内返回结果,将该测试忽略即可;

oceanbase_tpch_script.zip (15877031 KB)

1 个赞

oceanbase_tpch_30000_oceanbase_v32_fdr_2021-05-19_v01.pdf (962.6 KB)
FDR有同学反馈下载不了,我这里下载了一份作为附件上传

您好,我跟随这篇文章的流程进行tpch的ACID测试时出现了failed的结果,环境是docker下的oceanbasece:latest,单机模式下使用MINI_MODE启动,过程的打印结果如下:

[root@362f652c95b2 scripts]# sh run_acid.sh 127.0.0.1 2881 ./ 1 10 ./ ./
TPC Benchmark H - ACID-Test
Test Start Time=>05/23/2023  6:33:04.015
===========================

creating history table
ERROR 4179 (HY000) at line 8: range partition number not match not allowed

**************
ATOMICITY_TEST
**************
Test Start Time=>05/23/2023  6:33:04.083
ERROR 1146 (42S02) at line 1: Table 'test.history' doesn't exist
/root/oceanbase_tpch/benchmark_scripts/KIT//ACID/scripts/transaction_atomicity/acidtransaction 127.0.0.1 2881 test COMMIT NOWAIT .//ACID/SF1_U10_2023-05-23_0633/atomicity/orderkeys_1.log .//ACID/SF1_U10_2023-05-23_0633/atomicity/atomicity1_return_transaction.log root@test tpcuser .//ACID/SF1_U10_2023-05-23_0633/atomicity/atomicity1_values.log
/root/oceanbase_tpch/benchmark_scripts/KIT//ACID/scripts/atomicity_test_mainpart.sh: line 27:  5921 Segmentation fault      (core dumped) $ACIDDIR/scripts/transaction_atomicity/acidtransaction $ROOTHOST $PORT $DBNAME $ACTION NOWAIT $LOGDIR/orderkeys_$ACIDNUMBER.log $LOGDIR/atomicity${ACIDNUMBER}_return_transaction.log $TPC_USER_WITH_TENANT tpcuser $LOGDIR/atomicity${ACIDNUMBER}_values.log >> $LOGFILE
ERROR 1146 (42S02) at line 1: Table 'test.history' doesn't exist
Atomicity Test 1 failed (0 differences in diff file which very probably means no lines where changed).
ERROR 1146 (42S02) at line 1: Table 'test.history' doesn't exist
/root/oceanbase_tpch/benchmark_scripts/KIT//ACID/scripts/transaction_atomicity/acidtransaction 127.0.0.1 2881 test ROLLBACK NOWAIT .//ACID/SF1_U10_2023-05-23_0633/atomicity/orderkeys_2.log .//ACID/SF1_U10_2023-05-23_0633/atomicity/atomicity2_return_transaction.log root@test tpcuser .//ACID/SF1_U10_2023-05-23_0633/atomicity/atomicity2_values.log
/root/oceanbase_tpch/benchmark_scripts/KIT//ACID/scripts/atomicity_test_mainpart.sh: line 27:  5991 Segmentation fault      (core dumped) $ACIDDIR/scripts/transaction_atomicity/acidtransaction $ROOTHOST $PORT $DBNAME $ACTION NOWAIT $LOGDIR/orderkeys_$ACIDNUMBER.log $LOGDIR/atomicity${ACIDNUMBER}_return_transaction.log $TPC_USER_WITH_TENANT tpcuser $LOGDIR/atomicity${ACIDNUMBER}_values.log >> $LOGFILE
ERROR 1146 (42S02) at line 1: Table 'test.history' doesn't exist
Atomicity Test 2 passed (0 differences in diff file)
Atomicity test failed
Test END Time=>05/23/2023  6:33:08.366

****************
CONSISTENCY TEST
****************
Test Start Time=>05/23/2023  6:33:08.381
------------------
Consistency test 1
------------------
generate random orderkeys
creating  random transactions for stream 0
creating  random transactions for stream 1
creating  random transactions for stream 2
creating  random transactions for stream 3
creating  random transactions for stream 4
creating  random transactions for stream 5
creating  random transactions for stream 6
creating  random transactions for stream 7
creating  random transactions for stream 8
creating  random transactions for stream 9
creating  random transactions for stream 10

checking initial consistency
Consistency Test 1 failed ( Initial consistency check failed ).
Consistency test failed.
Test END Time=>05/23/2023  6:33:57.052
**************
ISOLATION TEST
**************
Test Start Time=>05/23/2023  6:33:57.065
----------------
Isolation test 1
----------------
Step 1: ACID query
Step 2: ACID transaction
Step 3: Same ACID query again
Step 4: Commit of ACID transaction
Step 5: Same ACID query again
Failed: Results are the same as those from the second query.
Isolation test 1 failed
----------------
Isolation test 2
----------------
Step 1: ACID query
Step 2: ACID transaction
Step 3: Same ACID query again
Step 4: Rollback of ACID transaction
Step 5: Same ACID query again
Isolation test 2 passed
----------------
Isolation test 3
----------------
Step 1: ACID query
Step 2: ACID transaction 1 until commit
Step 3: ACID transaction 2 with COMMIT 
Step 4: Commit of ACID transaction 1
Step 5: verify
Isolation test 3 failed.
 Results compare
----------------
Isolation test 4
----------------
Step 1: ACID transaction 1 until commit
Step 2: ACID transaction 2 with COMMIT
Step 3: Rollback of ACID transaction 1
Step 4: verify
Isolation test 4 failed.
 Results compare
----------------
Isolation test 5
----------------
118057
5591
Will use randomly selected values PS_PARTKEY=118057 and PS_SUPPKEY=5591 for this test
Step 1: ACID query
Step 2: ACID transaction
Step 3: Return all columns of the PARTSUPP-table for a random choice of PS_PARTKEY=118057 and PS_SUPPKEY=5591
test.partsupp
118057
--------------
select * from test.partsupp where PS_PARTKEY=118057 and PS_SUPPKEY=5591
--------------

+------------+------------+-------------+---------------+-----------------------------------------------------------------------------------------------------------------+
         ps_partkey      ps_suppkey      ps_availqty     ps_supplycost   ps_comment                                                                                                      
+------------+------------+-------------+---------------+-----------------------------------------------------------------------------------------------------------------+
             118057            5591             2981               905    even dolphins: regular deposits haggle above the carefully express deposits. regular, ironic instructions cajo 
+------------+------------+-------------+---------------+-----------------------------------------------------------------------------------------------------------------+
1 row in set (0.000 sec)

Bye
Step 4: Commit of ACID transaction
Step 5: Same ACID query again
Failed: Results are the same as those from the first query.
Isolation test 5 failed
----------------
Isolation test 6
----------------
Step 0: ACID query
Step 1: TPC-H query 1 with DELTA=0
Step 2: ACID transaction
/root/oceanbase_tpch/benchmark_scripts/KIT//ACID/scripts/isolation6.sh: line 69: [: Q1: integer expression expected
/root/oceanbase_tpch/benchmark_scripts/KIT//ACID/scripts/isolation6.sh: line 74: [: Q1: integer expression expected
Step 3: ACID query again
Failed: Results are the same as those from the first query
Isolation test 6 failed.
 Results compare
Isolation test failed. See .//ACID/SF1_U10_2023-05-23_0633/isolation/isolation.log for details.
Test End Time=>05/23/2023  6:37:48.278

====================
Testresult ACID Test
====================
ACID test failed
Test END Time=>05/23/2023  6:37:48.288

除了开头creating history table时的ERROR 4179 (HY000)外,后面的测试基本都有报failed,包括Consistency test的Initial consistency check failed和isolation测试中的Results compare。请问该如何排查?
注:直接测试的情况下会有libmysqlclient.so.20: cannot open shared object file: No such file or directory的错误,重新编译 acid的bin后该问题消失,但还是会有上述的错误

哈喽,这个脚本在最新的master分支代码上的确无法直接运行
问题: 碰到了一开头的4179


可以看到提示是和tablegroup的partition数不一样了

我的数据导入是使用obd test tpch完成的,这个tablegroup的partition数与acid 脚本中定义的不对等

在相对目录benchmark_scripts/KIT/ACID/sql/create_history_table_mysql.sql查看create table history

因此只要把这个脚本修改成16(你需要确认你的schema具体是多少,修改成相应的值)

另外还有个问题consistency检查会显示没过,简单看了一下,当时的测试公式是在decimal的定义下写的,目前obd test tpch导入的orders表中是定义为bitint, 计算过程会转为decimal发生精度损失

这是我在master分支上本地跑的情况