ERROR 4638 (HY000): The RootServer is not the master

OceanBase 技术问题
#1

【 使用环境 】生产环境
【问题描述】升级数据库表字段报错
报错：ERROR 4638 (HY000): The RootServer is not the master

3 个赞
#3

麻烦详提供以下信息：
1.从什么版本升级到什么版本？
2.使用什么工具升级（ocp or obd）？
3.OB集群部署架构是什么（1-1-1 或是其他）？
4.发下报错时 observer.log，rootservice.log，election.log
5.建议使用obdiag收集下切主问题信息

切主问题信息收集，默认不需要添加任何参数的话收集最近30分钟的信息

obdiag gather scene run --scene=observer.rootservice_switch
–config db_host=xx.xx.xx.xx
–config db_port=xxxx
–config tenant_sys.user=root@sys
–config tenant_sys.password=***
–config obcluster.servers.global.ssh_username=test
–config obcluster.servers.global.ssh_password=******
–config obcluster.servers.global.home_path=/home/admin/oceanbase

https://www.oceanbase.com/docs/common-obdiag-cn-1000000001326868

3 个赞
#4

升级表字段时报错
例如 ALTER TABLE sys_admin_info MODIFY password VARCHAR(512);
就会提示报错 之前一直都是好好的现在突然这样了

2 个赞
#5

菜鸟求助，执行上sql提示：
obclient> ALTER TABLE sys_admin_rule ADD COLUMN enc_role_name VARCHAR(125) ;
ERROR 4638 (HY000): The RootServer is not the master

2 个赞
ob数据库添加字段失败
#6

了解了，你的意思是执行这条DDL时报这个错，不是数据库版本升级的问题，
你发下你的OB数据库版本和部署架构吧

2 个赞
#7

5.7.25-OceanBase_CE-v4.2.1.1

2 个赞
#8

observer.log，rootservice.log也发下吧，通常在 /home/admin/oceanbase/log

2 个赞
#9

一直处于滚动追加无法捕获到 有关键性的 日志

2 个赞
#10

这两个文件可以打包压缩通过附件上传到这里

2 个赞
#11

rootservice.zip (48.2 MB)

2 个赞
#12

OB的部署架构是单节点吗？

另外查下 租户资源配置
SELECT c.TENANT_ID, e.TENANT_NAME, concat(c.NAME, ': ', d.NAME)pool:conf,concat(c.UNIT_COUNT, ’ unit: ', d.min_cpu, ‘C/’, ROUND(d.MEMORY_SIZE/1024/1024/1024,0), “G”) unit_info FROM DBA_OB_RESOURCE_POOLS c, DBA_OB_UNIT_CONFIGS d, DBA_OB_TENANTS e WHERE c.UNIT_CONFIG_ID=d.UNIT_CONFIG_ID AND c.TENANT_ID=e.TENANT_ID AND c.TENANT_ID>1000 ORDER BY c.TENANT_ID;

2 个赞
#14

obclient [oceanbase]> SELECT c.TENANT_ID, e.TENANT_NAME, CONCAT(c.NAME, ': ', d.NAME)conf,
→ CONCAT(c.UNIT_COUNT, ’ unit: ', d.min_cpu, ‘C/’, ROUND(d.MEMORY_SIZE/1024/1024/1024,0), ‘G’)
→ unit_info FROM DBA_OB_RESOURCE_POOLS c, DBA_OB_UNIT_CONFIGS d, DBA_OB_TENANTS e
→ WHERE c.UNIT_CONFIG_ID=d.UNIT_CONFIG_ID AND c.TENANT_ID=e.TENANT_ID AND c.TENANT_ID>1000 ORDER BY c.TENANT_ID;
±----------±------------±-------------±--------------+
| TENANT_ID | TENANT_NAME | conf | unit_info |
±----------±------------±-------------±--------------+
| 1002 | cbkj_tenant | pool1: unit1 | 1 unit: 1C/2G |
±----------±------------±-------------±--------------+
1 row in set (0.055 sec)

#15

sql.zip (1.7 KB)

上传到附件了，你再执行下

1 个赞
#16

obclient [oceanbase]> SELECT c.TENANT_ID, e.TENANT_NAME, concat(c.NAME, ': ', d.NAME)pool:conf,concat(c.UNIT_COUNT, ’ unit: ', d.min_cpu, ‘C/’, ROUND(d.MEMORY_SIZE/1024/1024/1024,0), “G”) unit_info FROM DBA_OB_RESOURCE_POOLS c, DBA_OB_UNIT_CONFIGS d, DBA_OB_TENANTS e WHERE c.UNIT_CONFIG_ID=d.UNIT_CONFIG_ID AND c.TENANT_ID=e.TENANT_ID AND c.TENANT_ID>1000 ORDER BY c.TENANT_ID;
±----------±------------±-------------±--------------+
| TENANT_ID | TENANT_NAME | pool:conf | unit_info |
±----------±------------±-------------±--------------+
| 1002 | cbkj_tenant | pool1: unit1 | 1 unit: 1C/2G |
±----------±------------±-------------±--------------+
1 row in set (0.033 sec)

obclient [oceanbase]> select /*+ READ_CONSISTENCY(WEAK) */
→ a.zone,
→ a.svr_ip,
→ a.svr_ip,
→ b.status,
→ cpu_capacity,
→ cpu_assigned_max,
→ cpu_capacity-cpu_assigned_max as cpu_free,
→ round(memory_limit /1024/1024/1024 ,2) as memory_total_gb,
→ round((memory_limit-mem_capacity) /1024/1024/1024 ,2) as system_memory_gb,
→ round(mem_assigned /1024/1024/1024 ,2) as mem_assigned_gb,
→ round((mem_capacity-mem_assigned) /1024/1024/1024 ,2) as memory_free_gb,
→ round(log_disk_capacity /1024/1024/1024 ,2) as log_disk_capacity_gb,
→ round(log_disk_assigned /1024/1024/1024 ,2) as log_disk_assigned_gb,
→ round((log_disk_capacity-log_disk_assigned) /1024/1024/1024 ,2) as log_disk_free_gb,
→ round((data_disk_capacity /1024/1024/1024 ),2) as data_disk_gb,
→ round((data_disk_in_use /1024/1024/1024 ),2) as data_disk_used_gb,
→ round((data_disk_capacity-data_disk_in_use) /1024/1024/1024 ,2) as data_disk_free_gb
→ from gv$ob_servers a
→ join oceanbase.DBA_OB_SERVERS b on a.zone=b.zone\G;
ERROR 4012 (HY000): Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again.
ERROR:
No query specified

1 个赞
#17

查询下如下SQL

select * from dba_ob_servers;
select * from dba_ob_zones;

1 个赞
#18

obclient [oceanbase]> select * from dba_ob_servers;
±----------±---------±—±------±---------±----------------±-------±---------------------------±----------±----------------------±---------------------------±---------------------------±------------------------------------------------------------------------------------------±------------------+
| SVR_IP | SVR_PORT | ID | ZONE | SQL_PORT | WITH_ROOTSERVER | STATUS | START_SERVICE_TIME | STOP_TIME | BLOCK_MIGRATE_IN_TIME | CREATE_TIME | MODIFY_TIME | BUILD_VERSION | LAST_OFFLINE_TIME |
±----------±---------±—±------±---------±----------------±-------±---------------------------±----------±----------------------±---------------------------±---------------------------±------------------------------------------------------------------------------------------±------------------+
| 127.0.0.1 | 2882 | 1 | zone2 | 2881 | YES | ACTIVE | 2024-08-22 09:45:06.043962 | NULL | NULL | 2024-05-23 11:05:11.226455 | 2024-09-02 09:25:57.260257 | 4.2.1.1_101000062023110109-7cb0fb494fce3962ae3f067991d6581c96bc0a9e(Nov 1 2023 09:55:42) | NULL |
±----------±---------±—±------±---------±----------------±-------±---------------------------±----------±----------------------±---------------------------±---------------------------±------------------------------------------------------------------------------------------±------------------+

1 个赞
#19

obclient [oceanbase]> select * from dba_ob_zones;
±------±---------------------------±---------------------------±-------±----±-----------±----------+
| ZONE | CREATE_TIME | MODIFY_TIME | STATUS | IDC | REGION | TYPE |
±------±---------------------------±---------------------------±-------±----±-----------±----------+
| zone2 | 2024-05-23 11:05:09.917810 | 2024-05-23 11:05:09.918899 | ACTIVE | | sys_region | ReadWrite |
±------±---------------------------±---------------------------±-------±----±-----------±----------+
1 row in set (0.069 sec)

#20

部署架构是单节点单副本，我们分析下日志，有进展尽快回复你

1 个赞
#21

麻烦了

#22

查询一下 这个几个信息
查看当前 leader

select * from oceanbase.__all_virtual_ha_diagnose where tenant_id=xxxx and ls_id=xxxx;
查看历史 leader

1. 选举层 leader 。

SELECT * FROM oceanbase.__all_server_event_history WHERE module=“election” AND value1=$tenant_id AND value2=$ls_id ORDER BY gmt_create desc limit 20;

2. PALF 层 leader 。

SELECT * FROM __all_server_event_history WHERE module=“log” AND value1=$tenant_id AND value2=$ls_id AND event like “%ROLE TRANSITION%” ORDER BY gmt_create desc limit 20;

1 个赞