租户资源扩展失败,求排查思路

【 使用环境 】测试环境
【 OB or 其他组件 】 OBserver
【 使用版本 】 4.3
【问题描述】给租户扩展资源时,提示server not enough to hold 1 new unit, please check zone valid servers,但是检查后发现空间是 充足的

obclient [oceanbase]> select a.zone,concat(a.svr_ip,':',a.svr_port) observer,a.CPU_CAPACITY cpu_total, (CPU_CAPACITY-cpu_assigned) cpu_free,                  round(a.memory_limit/1024/1024/1024 ) mem_total_gb, round((memory_limit-mem_assigned)/1024/1024/1024) mem_free_gb,  round(a.LOG_DISK_CAPACITY/1024/1024/1024) logdisk_total, round((a.LOG_DISK_CAPACITY-a.LOG_DISK_ASSIGNED)/1024/1024/1024) logdisk_free ,  round(a.DATA_DISK_CAPACITY/1024/1024/1024) data_total ,round((DATA_DISK_CAPACITY-a.DATA_DISK_IN_USE)/1024/1024/1024) data_free , b.status,usec_to_time(b.start_service_time) start_service_time,substr(b.build_version,1,10) as version   from GV$OB_SERVERS a join __all_server b on (a.svr_ip=b.svr_ip and a.svr_port=b.svr_port)  order by a.zone, a.svr_ip;
+-------+---------------------+-----------+----------+--------------+-------------+---------------+--------------+------------+-----------+--------+----------------------------+------------+
| zone  | observer            | cpu_total | cpu_free | mem_total_gb | mem_free_gb | logdisk_total | logdisk_free | data_total | data_free | status | start_service_time         | version    |
+-------+---------------------+-----------+----------+--------------+-------------+---------------+--------------+------------+-----------+--------+----------------------------+------------+
| zone1 | 192.168.122.10:2882 |        16 |       14 |           10 |           7 |             8 |            4 |          6 |         6 | ACTIVE | 2024-04-15 19:51:24.604539 | 4.3.0.1_10 |
| zone2 | 192.168.122.11:2882 |        16 |       14 |           10 |           7 |             8 |            4 |          6 |         6 | ACTIVE | 2024-04-15 19:51:25.874244 | 4.3.0.1_10 |
| zone3 | 192.168.122.12:2882 |        16 |       14 |           10 |           7 |             8 |            4 |          6 |         6 | ACTIVE | 2024-04-15 19:51:25.054977 | 4.3.0.1_10 |
+-------+---------------------+-----------+----------+--------------+-------------+---------------+--------------+------------+-----------+--------+----------------------------+------------+
3 rows in set (0.006 sec)

obclient [oceanbase]> select ZONE,SVR_IP,NAME,VALUE from GV$OB_PARAMETERS where SVR_IP='192.168.122.10' and name in ('log_disk_size','memory_limit','system_memory');
+-------+----------------+---------------+-------+
| ZONE  | SVR_IP         | NAME          | VALUE |
+-------+----------------+---------------+-------+
| zone1 | 192.168.122.10 | log_disk_size | 8G    |
| zone1 | 192.168.122.10 | system_memory | 1G    |
| zone1 | 192.168.122.10 | memory_limit  | 10G   |
+-------+----------------+---------------+-------+
3 rows in set (0.015 sec)

obclient [oceanbase]> select * from DBA_OB_RESOURCE_POOLS;
+------------------+----------+-----------+----------------------------+----------------------------+------------+----------------+-------------------+--------------+
| RESOURCE_POOL_ID | NAME     | TENANT_ID | CREATE_TIME                | MODIFY_TIME                | UNIT_COUNT | UNIT_CONFIG_ID | ZONE_LIST         | REPLICA_TYPE |
+------------------+----------+-----------+----------------------------+----------------------------+------------+----------------+-------------------+--------------+
|                1 | sys_pool |         1 | 2024-04-15 11:44:41.441110 | 2024-04-15 11:44:41.458155 |          1 |              1 | zone1;zone2;zone3 | FULL         |
|             1012 | t_pool   |      1018 | 2024-04-15 18:22:07.828188 | 2024-04-15 18:23:08.269565 |          1 |           1001 | zone1;zone2;zone3 | FULL         |
+------------------+----------+-----------+----------------------------+----------------------------+------------+----------------+-------------------+--------------+
2 rows in set (0.001 sec)

obclient [oceanbase]> select * from DBA_OB_UNIT_CONFIGS;
+----------------+-----------------+----------------------------+----------------------------+---------+---------+-------------+---------------+---------------------+---------------------+-------------+
| UNIT_CONFIG_ID | NAME            | CREATE_TIME                | MODIFY_TIME                | MAX_CPU | MIN_CPU | MEMORY_SIZE | LOG_DISK_SIZE | MAX_IOPS            | MIN_IOPS            | IOPS_WEIGHT |
+----------------+-----------------+----------------------------+----------------------------+---------+---------+-------------+---------------+---------------------+---------------------+-------------+
|              1 | sys_unit_config | 2024-04-15 11:44:41.435521 | 2024-04-15 19:57:49.209274 |       1 |       1 |  2147483648 |    2147483648 | 9223372036854775807 | 9223372036854775807 |           3 |
|           1001 | u_test          | 2024-04-15 11:48:19.770435 | 2024-04-15 11:48:19.770435 |       1 |       1 |  1073741824 |    2147483648 |               10000 |               10000 |           1 |
+----------------+-----------------+----------------------------+----------------------------+---------+---------+-------------+---------------+---------------------+---------------------+-------------+
2 rows in set (0.002 sec)

obclient [oceanbase]> ALTER RESOURCE TENANT tu1 UNIT_NUM = 2;
ERROR 4734 (HY000): zone 'zone1' server not enough to hold 1 new unit, please check zone valid servers

同一个unit一个observer上只能有一个,同一个zone里面可以有多个unit,你每个zone只有一台observer,所以只能有一个unit_num,你可以扩展unit的大小而不是改数量。

哦哦 明白了 ,醍醐灌顶啊~~ 多谢大佬 !!!

如果需要把unit_num改为2,你需要在每个zone里面再加一台observer的

1 个赞

哦 大佬 ,我多问语一句哈,那如何我 zone 里有俩节点,但是我资源池就给他分一个unit 行么?

可以的. unit_num =1 unit_num =2 .

多余节点不干活。但是可以失败转移吧

开启负载均衡的情况下都会干活的,租户的一部分对象会迁移到另一个observer上。

哦 明白了 ,多谢两位大佬