【 使用环境 】测试环境
【 OB or 其他组件 】OB
【 使用版本 】4.3.1
【问题描述】重置proxyro账号密码后，集群多处显示连接异常
【复现路径】
1.登录集群sys，执行重置proxyro账号的密码；
2.执行obd cluster display obdemo,显示 obagent is stopped
[root@ob01 conf]# obd cluster display obdemo
Get local repositories and plugins ok
Open ssh connection ok
Cluster status check ok
Connect to observer 10.0.22.41:2881 ok
Wait for observer init ok
±---------------------------------------------+
| observer |
±-----------±--------±-----±------±-------+
| ip | version | port | zone | status |
±-----------±--------±-----±------±-------+
| 10.0.22.41 | 4.3.0.1 | 2881 | zone1 | ACTIVE |
| 10.0.22.42 | 4.3.0.1 | 2881 | zone2 | ACTIVE |
| 10.0.22.43 | 4.3.0.1 | 2881 | zone3 | ACTIVE |
±-----------±--------±-----±------±-------+
obclient -h10.0.22.41 -P2881 -uroot -p’8wqBq4XBVYzHRocHAeQQ’ -Doceanbase -A
Connect to obproxy ok
±---------------------------------------------+
| obproxy |
±-----------±-----±----------------±-------+
| ip | port | prometheus_port | status |
±-----------±-----±----------------±-------+
| 10.0.22.42 | 2883 | 2884 | active |
±-----------±-----±----------------±-------+
obclient -h10.0.22.42 -P2883 -uroot@proxysys -p’7Az9qLt3Jt’ -Doceanbase -A
[WARN] server1(10.0.22.41) obagent is stopped
[WARN] server2(10.0.22.42) obagent is stopped
[WARN] server3(10.0.22.43) obagent is stopped
Connect to ocp-express ok
±--------------------------------------------------------------+
| ocp-express |
±-----------------------±---------±-----------------±-------+
| url | username | initial password | status |
±-----------------------±---------±-----------------±-------+
| http://10.0.22.43:8180 | admin | Ly5+C9r_ | active |
±-----------------------±---------±-----------------±-------+
Trace ID: 8b4a2dca-22fb-11ef-bdbe-00505686bc68
If you want to view detailed obd logs, please run: obd display-trace 8b4a2dca-22fb-11ef-bdbe-00505686bc68
[root@ob01 conf]#
[root@ob01 conf]#
3.由于没有单独重启obproxy的功能，遂采取重启整个集群的方式，重启后启动ocp-express出现连接不上meta db
[root@ob01 conf]# obd cluster stop obdemo
Get local repositories ok
Search plugins ok
Open ssh connection ok
Stop observer ok
Stop obshell ok
Stop obproxy ok
Stop obagent ok
Stop ocp-express ok
obdemo stopped
Trace ID: b3392d08-22fc-11ef-9076-00505686bc68
If you want to view detailed obd logs, please run: obd display-trace b3392d08-22fc-11ef-9076-00505686bc68
[root@ob01 conf]# obd cluster start obdemo
Get local repositories ok
Search plugins ok
Load cluster param plugin ok
Open ssh connection ok
Check before start observer ok
Check before start obproxy ok
Check before start obagent ok
Check before start ocp-express ok
Start observer ok
observer program health check ok
obshell program health check ok
Connect to observer 10.0.22.41:2881 ok
Start obproxy ok
obproxy program health check ok
Connect to obproxy ok
Initialize obproxy-ce ok
Start obagent ok
obagent program health check ok
Connect to Obagent ok
Start ocp-express x
[ERROR] 10.0.22.43: failed to connect meta db
[ERROR] ocp-express start failed
Wait for observer init ok
±---------------------------------------------+
| observer |
±-----------±--------±-----±------±-------+
| ip | version | port | zone | status |
±-----------±--------±-----±------±-------+
| 10.0.22.41 | 4.3.0.1 | 2881 | zone1 | ACTIVE |
| 10.0.22.42 | 4.3.0.1 | 2881 | zone2 | ACTIVE |
| 10.0.22.43 | 4.3.0.1 | 2881 | zone3 | ACTIVE |
±-----------±--------±-----±------±-------+
obclient -h10.0.22.41 -P2881 -uroot -p’8wqBq4XBVYzHRocHAeQQ’ -Doceanbase -A
±---------------------------------------------+
| obproxy |
±-----------±-----±----------------±-------+
| ip | port | prometheus_port | status |
±-----------±-----±----------------±-------+
| 10.0.22.42 | 2883 | 2884 | active |
±-----------±-----±----------------±-------+
obclient -h10.0.22.42 -P2883 -uroot@proxysys -p’7Az9qLt3Jt’ -Doceanbase -A
±--------------------------------------------------------------+
| obagent |
±-----------±-------------------±-------------------±-------+
| ip | mgragent_http_port | monagent_http_port | status |
±-----------±-------------------±-------------------±-------+
| 10.0.22.41 | 8089 | 8088 | active |
| 10.0.22.42 | 8089 | 8088 | active |
| 10.0.22.43 | 8089 | 8088 | active |
±-----------±-------------------±-------------------±-------+
See OceanBase分布式数据库-海量数据 笔笔算数 .
Trace ID: c5f0cafa-22fc-11ef-9402-00505686bc68
If you want to view detailed obd logs, please run: obd display-trace c5f0cafa-22fc-11ef-9402-00505686bc68
[root@ob01 conf]#
4.执行 obd display-trace c5f0cafa-22fc-11ef-9402-00505686bc68 查看日志
…
[2024-06-05 13:47:17.023] [DEBUG] – connect 10.0.22.42 -P2883 -umeta@ocp -p2KExFJcwxQ
[2024-06-05 13:47:19.023] [DEBUG] – connect 10.0.22.42 -P2883 -umeta@ocp -p2KExFJcwxQ
[2024-06-05 13:47:21.020] [DEBUG] – connect 10.0.22.42 -P2883 -umeta@ocp -p2KExFJcwxQ
[2024-06-05 13:47:23.021] [DEBUG] – connect 10.0.22.42 -P2883 -umeta@ocp -p2KExFJcwxQ
[2024-06-05 13:47:25.025] [DEBUG] – connect 10.0.22.42 -P2883 -umeta@ocp -p2KExFJcwxQ
[2024-06-05 13:47:27.024] [DEBUG] – connect 10.0.22.42 -P2883 -umeta@ocp -p2KExFJcwxQ
[2024-06-05 13:47:29.025] [DEBUG] – connect 10.0.22.42 -P2883 -umeta@ocp -p2KExFJcwxQ
[2024-06-05 13:47:31.033] [ERROR] 10.0.22.43: failed to connect meta db
[2024-06-05 13:47:31.056] [INFO] [ERROR] 10.0.22.43: failed to connect meta db
5.使用root用户登录proxy正常
[root@ob01 log]# obclient -h10.0.22.42 -P2883 -uroot@proxysys -p’7Az9qLt3Jt’ -Doceanbase -A
Welcome to the OceanBase. Commands end with ; or \g.
Your OceanBase connection id is 443809803
Server version: 5.6.25
Copyright (c) 2000, 2018, OceanBase and/or its affiliates. All rights reserved.
Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.
obclient [oceanbase]>
obclient [oceanbase]>
obclient [oceanbase]>
【附件及日志】推荐使用OceanBase敏捷诊断工具obdiag收集诊断信息,详情参见链接（右键跳转查看）：