为什么创建的租户,压测性能特别不稳定?

【 使用环境 】测试环境
【 OB or 其他组件 】

|sysbench|172.16.11.134 → 172.16.11.134:2883|
|obproxy|172.16.11.134:2883|
|observer zone1|172.16.11.134:2881|
|observer zone2|172.16.11.135:2881|
|observer zone3|172.16.11.136:2881|
【 使用版本 】OB version 4.0.0.0
【问题描述】

cat sysbench_config_ob
mysql-host=172.16.11.134
mysql-port=2883 # obproxy端口
mysql-user=root@tenant1
mysql-password=''
mysql-db=test
time=600
threads=10
report-interval=1
db-driver=mysql


大概就是按照官网创建的租户
create resource unit sysbench_unit max_cpu 4, max_memory 6000000000, max_iops 128, max_disk_size 53687091200, max_session_num 64, MIN_CPU=26, MIN_MEMORY=60000000000, MIN_IOPS=128;
create resource pool sysbench_pool unit = 'sysbench_unit', unit_num = 1, zone_list=('zone1');
create tenant tenant1 resource_pool_list=('sysbench_pool'), charset=utf8mb4, replica_num=3, zone_list('zone1', 'zone2', 'zone3'), primary_zone=RANDOM, locality='F@zone1,F@zone2,F@zone3' set variables ob_compatibility_mode='mysql', ob_tcp_invited_nodes='%';

ALTER resource unit test_unit min_cpu=2,max_cpu=2, max_memory='6G',min_memory='3G', max_disk_size='500G',max_iops=10000,min_iops=10000,max_session_num=10000; 

性能不稳定,一会高一会低,用mysql测平稳

[ 110s ] thds: 10 tps: 1092.03 qps: 1092.03 (r/w/o: 0.00/1092.03/0.00) lat (ms,95%): 14.21 err/s: 0.00 reconn/s: 0.00
[ 111s ] thds: 10 tps: 1130.27 qps: 1130.27 (r/w/o: 0.00/1130.27/0.00) lat (ms,95%): 13.95 err/s: 0.00 reconn/s: 0.00
[ 112s ] thds: 10 tps: 1160.06 qps: 1160.06 (r/w/o: 0.00/1160.06/0.00) lat (ms,95%): 13.46 err/s: 0.00 reconn/s: 0.00
[ 113s ] thds: 10 tps: 1143.73 qps: 1143.73 (r/w/o: 0.00/1143.73/0.00) lat (ms,95%): 12.98 err/s: 0.00 reconn/s: 0.00
[ 114s ] thds: 10 tps: 1087.05 qps: 1087.05 (r/w/o: 0.00/1087.05/0.00) lat (ms,95%): 14.73 err/s: 0.00 reconn/s: 0.00
[ 115s ] thds: 10 tps: 1112.77 qps: 1112.77 (r/w/o: 0.00/1112.77/0.00) lat (ms,95%): 13.95 err/s: 0.00 reconn/s: 0.00
[ 116s ] thds: 10 tps: 1130.11 qps: 1130.11 (r/w/o: 0.00/1130.11/0.00) lat (ms,95%): 13.22 err/s: 0.00 reconn/s: 0.00
[ 117s ] thds: 10 tps: 1150.67 qps: 1150.67 (r/w/o: 0.00/1150.67/0.00) lat (ms,95%): 13.22 err/s: 0.00 reconn/s: 0.00
[ 118s ] thds: 10 tps: 834.47 qps: 834.47 (r/w/o: 0.00/834.47/0.00) lat (ms,95%): 13.70 err/s: 0.00 reconn/s: 0.00
[ 119s ] thds: 10 tps: 2.00 qps: 2.00 (r/w/o: 0.00/2.00/0.00) lat (ms,95%): 831.46 err/s: 0.00 reconn/s: 0.00
[ 120s ] thds: 10 tps: 8.00 qps: 8.00 (r/w/o: 0.00/8.00/0.00) lat (ms,95%): 1938.16 err/s: 0.00 reconn/s: 0.00
[ 121s ] thds: 10 tps: 21.00 qps: 21.00 (r/w/o: 0.00/21.00/0.00) lat (ms,95%): 1479.41 err/s: 0.00 reconn/s: 0.00
[ 122s ] thds: 10 tps: 1118.84 qps: 1118.84 (r/w/o: 0.00/1118.84/0.00) lat (ms,95%): 12.98 err/s: 0.00 reconn/s: 0.00
[ 123s ] thds: 10 tps: 1747.24 qps: 1747.24 (r/w/o: 0.00/1747.24/0.00) lat (ms,95%): 10.65 err/s: 0.00 reconn/s: 0.00
[ 124s ] thds: 10 tps: 2046.68 qps: 2046.68 (r/w/o: 0.00/2046.68/0.00) lat (ms,95%): 7.56 err/s: 0.00 reconn/s: 0.00
[ 125s ] thds: 10 tps: 2094.84 qps: 2094.84 (r/w/o: 0.00/2094.84/0.00) lat (ms,95%): 7.70 err/s: 0.00 reconn/s: 0.00
[ 126s ] thds: 10 tps: 1779.81 qps: 1779.81 (r/w/o: 0.00/1779.81/0.00) lat (ms,95%): 12.30 err/s: 0.00 reconn/s: 0.00
[ 127s ] thds: 10 tps: 1665.98 qps: 1665.98 (r/w/o: 0.00/1665.98/0.00) lat (ms,95%): 17.32 err/s: 0.00 reconn/s: 0.00
[ 128s ] thds: 10 tps: 1759.31 qps: 1759.31 (r/w/o: 0.00/1759.31/0.00) lat (ms,95%): 17.32 err/s: 0.00 reconn/s: 0.00
[ 129s ] thds: 10 tps: 1782.62 qps: 1782.62 (r/w/o: 0.00/1782.62/0.00) lat (ms,95%): 13.22 err/s: 0.00 reconn/s: 0.00
···


cd /root/.oceanbase-all-in-one/conf/
cp mini-distributed-with-obproxy-example.yaml  allen.yaml
修改以下的参数
vi allen.yaml
# user:  <<<<  如果需要使用其他用户部署ob,则修改下面的内容,测试使用了root 用户部署,这部分内容没有修改
#   username: username
#   password: password
#   key_file: your ssh-key file path if need
#   port: your ssh port, default 22
#   timeout: ssh connection timeout (second), default 30

oceanbase-ce:  # 在这里修改每个节点的server name 和 ip 地址。
  servers:
    - name: server1
      # Please don't use hostname, only IP can be supported
      ip: 172.16.11.134
    - name: server2
      ip: 172.16.11.135
    - name: server3
      ip: 172.16.11.136

#在这里修改集群级别的全局配置。
 global:
    # Please set devname as the network adaptor's name whose ip is  in the setting of severs.
    # if set severs as "127.0.0.1", please set devname as "lo"
    # if current ip is 192.168.1.10, and the ip's network adaptor's name is "eth0", please use "eth0"
    devname: eth0
    cluster_id: 1
    # please set memory limit to a suitable value which is matching resource.
    memory_limit: 24G # The maximum running memory for an observer
    system_memory: 1G # The reserved system memory. system_memory is reserved for general tenants. The default value is 30G.
    datafile_size: 16G # Size of the data file.
    log_disk_size: 16G # The size of disk space used by the clog files.
    cpu_count: 16
    production_mode: true
    syslog_level: INFO # System log level. The default value is INFO.
    enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true.
    enable_syslog_recycle: true # Enable auto system log recycling or not. The default value is false.
    max_syslog_file_count: 4 # The maximum number of reserved log files before enabling auto recycling. The default value is 0.
    # observer cluster name, consistent with obproxy's cluster_name
    appname: obcluster
    # root_password: # root user password, can be empty
    # proxyro_password: # proxyro user pasword, consistent with obproxy's observer_sys_password, can be empty
    
    obproxy-ce:
  # Set dependent components for the component.
  # When the associated configurations are not done, OBD will automatically get the these configurations from the dependent components.
  depends:
    - oceanbase-ce
  servers:
    - 172.16.11.134 #在这里输入obproxy(ob 自己做的一个load balancer) 的地址

资源太低,看看转储情况。写入太大,可能在转储

老师,请问如何看转储?
监控界面怎么打开?

大佬请教一下, 怎么排查是哪些资源不足呀