【 使用环境 】测试环境
【 使用版本 】oceanbase-all-in-one-4.3.5
【问题描述】
Check before start obagent ok
Check before start prometheus ok
Check before start grafana ok
cluster scenario: express_oltp
Start observer ok
observer program health check ok
Connect to observer 127.0.0.1:2881 ok
oceanbase bootstrap ok
obshell start ok
obshell program health check ok
obshell bootstrap x
[ERROR] obshell bootstrap failed: Task ‘Take over’ execution failed: 127.0.0.1:2886 ERROR: [MySQL.Error]: MySQL error: Error 4013 (HY001): No memory or reach tenant memory limit
See https://www.oceanbase.com/product/ob-deployer/error-codes .
Trace ID: c07dfa08-b076-11f0-bf8f-b04fa6e48b3f
If you want to view detailed obd logs, please run: obd display-trace c07dfa08-b076-11f0-bf8f-b04fa6e48b3f
【日志信息】
[2025-10-24 09:08:10.570] [INFO] obshell start
[2025-10-24 09:08:10.571] [DEBUG] – local execute: cat /root/oceanbase-ce/run/obshell.pid
[2025-10-24 09:08:10.573] [DEBUG] – exited code 1, error output:
[2025-10-24 09:08:10.573] [DEBUG] cat: /root/oceanbase-ce/run/obshell.pid: 没有那个文件或目录
[2025-10-24 09:08:10.574] [DEBUG]
[2025-10-24 09:08:10.574] [DEBUG] – root@127.0.0.1 export OB_ROOT_PASSWORD=******
[2025-10-24 09:08:10.574] [DEBUG] – start obshell: cd /root/oceanbase-ce; /root/oceanbase-ce/bin/obshell admin start --ip 127.0.0.1 --port 2886
[2025-10-24 09:08:10.574] [DEBUG] – local execute: cd /root/oceanbase-ce; /root/oceanbase-ce/bin/obshell admin start --ip 127.0.0.1 --port 2886
[2025-10-24 09:08:14.901] [DEBUG] – exited code 0
[2025-10-24 09:08:15.000] [INFO] obshell program health check
[2025-10-24 09:08:15.001] [DEBUG] – local execute: cat /root/oceanbase-ce/run/obshell.pid
[2025-10-24 09:08:15.003] [DEBUG] – exited code 0
[2025-10-24 09:08:15.003] [DEBUG] – Get 127.0.0.1 obshell[pid: 63560]
[2025-10-24 09:08:15.004] [DEBUG] – local execute: ls /proc/63560
[2025-10-24 09:08:15.006] [DEBUG] – exited code 0
[2025-10-24 09:08:15.006] [DEBUG] – 127.0.0.1 obshell[pid: 63560] started
[2025-10-24 09:08:15.131] [DEBUG] - sub obshell_start ref count to 0
[2025-10-24 09:08:15.131] [DEBUG] - export obshell_start
[2025-10-24 09:08:15.132] [DEBUG] - plugin oceanbase-ce-py_script_obshell_start-4.2.1.4 result: True
[2025-10-24 09:08:15.132] [DEBUG] - Searching obshell_bootstrap plugin for components …
[2025-10-24 09:08:15.132] [DEBUG] - Searching obshell_bootstrap plugin for oceanbase-ce-4.3.5.4-104000042025090916.el7-fc8a56acab02b985fe98535b6ce341a97603568b
[2025-10-24 09:08:15.133] [DEBUG] - Found for oceanbase-ce-py_script_obshell_bootstrap-4.2.1.4 for oceanbase-ce-4.3.5.4
[2025-10-24 09:08:15.133] [DEBUG] - Call plugin oceanbase-ce-py_script_obshell_bootstrap-4.2.1.4 for oceanbase-ce-4.3.5.4-104000042025090916.el7-fc8a56acab02b985fe98535b6ce341a97603568b
[2025-10-24 09:08:15.133] [DEBUG] - import obshell_bootstrap
[2025-10-24 09:08:15.213] [DEBUG] - add obshell_bootstrap ref count to 1
[2025-10-24 09:08:15.214] [INFO] obshell bootstrap
[2025-10-24 09:08:23.211] [ERROR] Traceback (most recent call last):
[2025-10-24 09:08:23.211] [ERROR] File “core.py”, line 1731, in demo
[2025-10-24 09:08:23.211] [ERROR] File “core.py”, line 2277, in start_cluster
[2025-10-24 09:08:23.211] [ERROR] File “core.py”, line 2348, in _start_cluster
[2025-10-24 09:08:23.211] [ERROR] File “core.py”, line 256, in run_workflow
[2025-10-24 09:08:23.211] [ERROR] File “core.py”, line 298, in run_plugin_template
[2025-10-24 09:08:23.212] [ERROR] File “core.py”, line 347, in call_plugin
[2025-10-24 09:08:23.212] [ERROR] File “_plugin.py”, line 348, in call
[2025-10-24 09:08:23.212] [ERROR] File “_plugin.py”, line 304, in _new_func
[2025-10-24 09:08:23.212] [ERROR] File “/root/.obd/plugins/oceanbase-ce/4.2.1.4/obshell_bootstrap.py”, line 54, in obshell_bootstrap
[2025-10-24 09:08:23.212] [ERROR] client.v1.wait_dag_succeed(dag.generic_id)
[2025-10-24 09:08:23.212] [ERROR] File “/usr/obd/lib/site-packages/obshell/service/client_v1.py”, line 1110, in wait_dag_succeed
[2025-10-24 09:08:23.212] [ERROR] raise TaskExecuteFailedError(f"{logs}", dag)
[2025-10-24 09:08:23.212] [ERROR] obshell.service.client_v1.TaskExecuteFailedError: Task ‘Take over’ execution failed: 127.0.0.1:2886 ERROR: [MySQL.Error]: MySQL error: Error 4013 (HY001): No memory or reach tenant memory limit
[2025-10-24 09:08:23.212] [ERROR]
[2025-10-24 09:08:23.212] [ERROR]
[2025-10-24 09:08:23.293] [ERROR] [ERROR] obshell bootstrap failed: Task ‘Take over’ execution failed: 127.0.0.1:2886 ERROR: [MySQL.Error]: MySQL error: Error 4013 (HY001): No memory or reach tenant memory limit
[2025-10-24 09:08:23.293] [ERROR]
[2025-10-24 09:08:23.293] [DEBUG] - sub obshell_bootstrap ref count to 0
[2025-10-24 09:08:23.293] [DEBUG] - export obshell_bootstrap
[2025-10-24 09:08:23.293] [DEBUG] - plugin oceanbase-ce-py_script_obshell_bootstrap-4.2.1.4 result: False
[2025-10-24 09:08:23.295] [INFO] See https://www.oceanbase.com/product/ob-deployer/error-codes .
[2025-10-24 09:08:23.296] [INFO] Trace ID: d367a480-b075-11f0-b071-b04fa6e48b3f
[2025-10-24 09:08:23.296] [INFO] If you want to view detailed obd logs, please run: obd display-trace d367a480-b075-11f0-b071-b04fa6e48b3f
[2025-10-24 09:08:23.298] [DEBUG] - unlock /root/.obd/lock/global
[2025-10-24 09:08:23.298] [DEBUG] - unlock /root/.obd/lock/deploy_demo
[2025-10-24 09:08:23.298] [DEBUG] - unlock /root/.obd/lock/mirror_and_repo