OceanBase Deploy: 2.7.0重启DB失败

OceanBase 技术问题
#1

【 使用环境 】测试环境
【 OB or 其他组件 】
【 使用版本 】OceanBase Deploy: 2.7.0
oceanbase0429日启动失败日志.txt (38.8 KB)

【问题描述】执行obd cluster start demo启动失败
【复现路径】obd cluster start demo
【附件及日志】推荐使用OceanBase敏捷诊断工具obdiag收集诊断信息,详情参见链接（右键跳转查看）：

【SOP系列 22 】——故障诊断第一步(自助诊断和诊断信息收集)

【备注】基于 LLM 和开源文档 RAG 的论坛小助手已开放测试，在发帖时输入 [@论坛小助手] 即可召唤小助手，欢迎试用！

2 个赞
#3

根据您提供的日志和问题描述，以下是可能的原因分析及解决方法：

1. 检查系统内核参数

  • 根据文档信息，如果操作系统的内核参数不在推荐范围内，可能会导致启动失败。测试环境中可以尝试跳过严格的内核参数检查。
  • 解决方法：
    • 如果是测试环境且没有权限修改内核参数，可以通过以下命令修改配置文件，将 production_mode 设置为 false 来跳过系统参数阻塞检查：
      obd cluster edit-config demo
      在配置文件中添加或修改以下内容：
      production_mode: false

2. 检查内存是否充足

  • 内存不足也可能导致启动失败。请确保您的可用内存满足最低要求（对于 OceanBase 数据库 4.x 版本，memory_limit 不低于 6G）。
  • 解决方法：
    • 尝试释放缓存内存：
      sudo sysctl -w vm.drop_caches=1
# 或
sudo echo 1 > /proc/sys/vm/drop_caches
    • 如果内存仍然不足，请调整 memory_limit 和 system_memory 参数。通常情况下，memory_limit/3 ≤ system_memory ≤ memory_limit/2。通过以下命令编辑配置文件：
      obd cluster edit-config demo
      修改相关参数以适应您的硬件资源。

3. 检查 observer 进程启动失败

  • 如果 observer 进程启动时内部组件或服务初始化失败，可能会导致启动失败。错误码 4395 表示此问题。
  • 解决方法：
    • 检查日志文件以获取更多详细信息。如果问题无法通过日志定位，请联系技术支持人员协助排查。

4. 验证集群状态

  • 确保集群的配置文件正确无误，并且所有依赖组件均已正确部署。
  • 解决方法：
    • 使用以下命令检查集群状态：
      obd cluster status demo
    • 如果发现某些组件未正常运行，可以尝试重新部署或修复集群：
      obd cluster redeploy demo

5. 检查版本兼容性

  • 确保使用的 obd 版本与 OceanBase 数据库版本兼容。您当前使用的是 OceanBase Deploy: 2.7.0 和 oceanbase-ce-4.3.0.1。
  • 解决方法：
    • 如果怀疑版本不兼容，可以尝试升级 obd 到最新版本：
      pip install --upgrade ob-deploy

总结

根据上述分析，请按照以下步骤逐步排查问题：

  1. 检查并调整系统内核参数（如果是测试环境，可设置 production_mode: false）。
  2. 确保内存充足，并调整 memory_limit 和 system_memory 参数。
  3. 检查 observer 进程启动日志，定位具体失败原因。
  4. 验证集群状态，必要时重新部署或修复集群。
  5. 确认 obd 和 OceanBase 数据库版本的兼容性。

如果以上方法仍无法解决问题，请提供更详细的日志信息，或者联系 OceanBase 的人工答疑以获取进一步支持。

详情请参考:

  1. OceanBase 安装部署工具 V2.8.0
  2. 错误码
  3. 通过命令行部署 oblogproxy
  4. OceanBase 安装部署工具 V1.3.3
  5. OceanBase 安装部署工具 V1.6.0
  6. 常见问题汇总
  7. 性能测试
  8. 4000 ~ 4499
2 个赞
#5

1、ob的版本号
2、obd cluster edit-config demo --保存到文本里
3、 obd日志： 默认保存在安装obd的用户home路径： cd ~/.obd/log/

2 个赞
#6

1、
obd --version
OceanBase Deploy: 2.7.0
REVISION: ed74f49b8568586166b8126f1d270cbe2a0e7be1
BUILD_BRANCH: HEAD
BUILD_TIME: Mar 27 2024 19:46:54OURCE
Copyright (C) 2021 OceanBase
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

2、
obd cluster edit-config demo
Search param plugin and load ok

obagent:
servers:

  • 127.0.0.1
    global:
    home_path: /root/obagent
    ob_monitor_status: active
    depends:
  • oceanbase-ce
    obproxy-ce:
    servers:
  • 127.0.0.1
    global:
    home_path: /root/obproxy-ce
    skip_proxy_sys_private_check: true
    enable_strict_kernel_release: false
    enable_cluster_checkout: false
    proxy_mem_limited: 500M
    127.0.0.1:
    proxy_id: 4671
    client_session_id_version: 2
    depends:
  • oceanbase-ce
    oceanbase-ce:
    servers:
  • 127.0.0.1
    global:
    home_path: /root/oceanbase-ce
    cluster_id: 1716372876
    enable_syslog_recycle: true
    enable_syslog_wf: false
    max_syslog_file_count: 4
    memory_limit: 6144M
    production_mode: false
    __min_full_resource_pool_memory: 1073741824
    system_memory: 1024M
    cpu_count: 34
    datafile_size: 2048M
    datafile_maxsize: 8192M
    datafile_next: 2048M
    “/tmp/tmpzggq6zq5.yaml” 55L, 1122C

3、
obdtoady.zip (12.0 KB)

2 个赞
#7

建议优先升级一下obd
ps -ef|grep observer看看集群起来没。
执行obd cluster start demo试一下能启动么。可能存在机器性能差导致安装部署启动超时问题

2 个赞
#8

用了一段时间了，以前也遇到过，执行obd cluster start demo --wop这个是可以起来的，现在不行

ps -ef|grep observer
root 13682 35222 0 11:18 pts/0 00:00:00 grep --color=auto observer

obd cluster start demo
Get local repositories ok
Search plugins ok
Load cluster param plugin ok
Open ssh connection ok
Check before start observer x
[ERROR] OBD-1001: 127.0.0.1:2886 port is already used, Please choose another unoccupied port or terminate the process occupying the port
[WARN] OBD-1007: (127.0.0.1) The recommended number of max user processes is 655350 (Current value: 120000)
[WARN] OBD-1017: (127.0.0.1) The value of the “vm.max_map_count” must be within [327600, 1310720] (Current value: 65530, Recommended value: 655360)

Check before start obagent ok
Check before start obproxy ok
Check before start prometheus ok
Check before start grafana ok
See https://www.oceanbase.com/product/ob-deployer/error-codes .
Trace ID: a6852d2e-24a8-11f0-a115-005056b7a1e2
If you want to view detailed obd logs, please run: obd display-trace a6852d2e-24a8-11f0-a115-005056b7a1e2

2 个赞
#9

image
image739×139 13.5 KB

端口被占用杀掉重启试试
麻烦提供一份observer日志吧。看看启动卡在哪里了

1 个赞
#10

obd display-trace 1ef13938-24a9-11f0-a7c8-005056b7a1e2
[2025-04-29 11:22:03.568] [DEBUG] - cmd: [‘demo’]
[2025-04-29 11:22:03.568] [DEBUG] - opts: {‘servers’: None, ‘components’: None, ‘force_delete’: None, ‘strict_check’: None, ‘without_parameter’: True}
[2025-04-29 11:22:03.569] [DEBUG] - mkdir /root/.obd/lock/
[2025-04-29 11:22:03.569] [DEBUG] - unknown lock mode
[2025-04-29 11:22:03.569] [DEBUG] - try to get share lock /root/.obd/lock/global
[2025-04-29 11:22:03.569] [DEBUG] - share lock /root/.obd/lock/global, count 1
[2025-04-29 11:22:03.569] [DEBUG] - Get Deploy by name
[2025-04-29 11:22:03.569] [DEBUG] - mkdir /root/.obd/cluster/
[2025-04-29 11:22:03.569] [DEBUG] - mkdir /root/.obd/config_parser/
[2025-04-29 11:22:03.570] [DEBUG] - try to get exclusive lock /root/.obd/lock/deploy_demo
[2025-04-29 11:22:03.570] [DEBUG] - exclusive lock /root/.obd/lock/deploy_demo, count 1
[2025-04-29 11:22:03.577] [DEBUG] - Deploy status judge
[2025-04-29 11:22:03.577] [INFO] Get local repositories
[2025-04-29 11:22:03.578] [DEBUG] - mkdir /root/.obd/repository
[2025-04-29 11:22:03.578] [DEBUG] - Get local repository obagent-4.2.2-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.578] [DEBUG] - Search repository obagent version: 4.2.2, tag: 19739a07a12eab736aff86ecf357b1ae660b554e, release: None, package_hash: None
[2025-04-29 11:22:03.579] [DEBUG] - try to get share lock /root/.obd/lock/mirror_and_repo
[2025-04-29 11:22:03.579] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo, count 1
[2025-04-29 11:22:03.579] [DEBUG] - mkdir /root/.obd/repository/obagent
[2025-04-29 11:22:03.581] [DEBUG] - Found repository obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.581] [DEBUG] - Get local repository obproxy-ce-4.2.3.0-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:03.581] [DEBUG] - Search repository obproxy-ce version: 4.2.3.0, tag: 0490ebc04220def8d25cb9cac9ac61a4efa6d639, release: None, package_hash: None
[2025-04-29 11:22:03.581] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo, count 2
[2025-04-29 11:22:03.581] [DEBUG] - mkdir /root/.obd/repository/obproxy-ce
[2025-04-29 11:22:03.583] [DEBUG] - Found repository obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:03.583] [DEBUG] - Get local repository oceanbase-ce-4.3.0.1-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:03.583] [DEBUG] - Search repository oceanbase-ce version: 4.3.0.1, tag: c4a03c83614f50c99ddb1c37dda858fa5d9b14b7, release: None, package_hash: None
[2025-04-29 11:22:03.583] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo, count 3
[2025-04-29 11:22:03.583] [DEBUG] - mkdir /root/.obd/repository/oceanbase-ce
[2025-04-29 11:22:03.585] [DEBUG] - Found repository oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:03.585] [DEBUG] - Get local repository grafana-7.5.17-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:03.585] [DEBUG] - Search repository grafana version: 7.5.17, tag: 1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6, release: None, package_hash: None
[2025-04-29 11:22:03.585] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo, count 4
[2025-04-29 11:22:03.586] [DEBUG] - mkdir /root/.obd/repository/grafana
[2025-04-29 11:22:03.587] [DEBUG] - Found repository grafana-7.5.17-1-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:03.588] [DEBUG] - Get local repository prometheus-2.37.1-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:03.588] [DEBUG] - Search repository prometheus version: 2.37.1, tag: 58913c7606f05feb01bc1c6410346e5fc31cf263, release: None, package_hash: None
[2025-04-29 11:22:03.588] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo, count 5
[2025-04-29 11:22:03.588] [DEBUG] - mkdir /root/.obd/repository/prometheus
[2025-04-29 11:22:03.590] [DEBUG] - Found repository prometheus-2.37.1-10000102022110211.el7-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:03.709] [DEBUG] - Get deploy config
[2025-04-29 11:22:03.727] [INFO] Search plugins
[2025-04-29 11:22:03.727] [DEBUG] - Searching start_check plugin for components …
[2025-04-29 11:22:03.727] [DEBUG] - Searching start_check plugin for obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.728] [DEBUG] - mkdir /root/.obd/plugins
[2025-04-29 11:22:03.728] [DEBUG] - Found for obagent-py_script_start_check-1.3.0 for obagent-4.2.2
[2025-04-29 11:22:03.729] [DEBUG] - Searching start_check plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:03.729] [DEBUG] - Found for obproxy-ce-py_script_start_check-4.2.3 for obproxy-ce-4.2.3.0
[2025-04-29 11:22:03.729] [DEBUG] - Searching start_check plugin for oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:03.729] [DEBUG] - Found for oceanbase-ce-py_script_start_check-4.2.2.0 for oceanbase-ce-4.3.0.1
[2025-04-29 11:22:03.729] [DEBUG] - Searching start_check plugin for grafana-7.5.17-1-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:03.729] [DEBUG] - Found for grafana-py_script_start_check-7.5.17 for grafana-7.5.17
[2025-04-29 11:22:03.729] [DEBUG] - Searching start_check plugin for prometheus-2.37.1-10000102022110211.el7-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:03.729] [DEBUG] - Found for prometheus-py_script_start_check-2.37.1 for prometheus-2.37.1
[2025-04-29 11:22:03.730] [DEBUG] - Searching create_tenant plugin for components …
[2025-04-29 11:22:03.730] [DEBUG] - Searching create_tenant plugin for obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.730] [DEBUG] - No such create_tenant plugin for obagent-4.2.2
[2025-04-29 11:22:03.730] [DEBUG] - Searching create_tenant plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:03.730] [DEBUG] - No such create_tenant plugin for obproxy-ce-4.2.3.0
[2025-04-29 11:22:03.730] [DEBUG] - Searching create_tenant plugin for oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:03.730] [DEBUG] - Found for oceanbase-ce-py_script_create_tenant-4.3.0.0 for oceanbase-ce-4.3.0.1
[2025-04-29 11:22:03.730] [DEBUG] - Searching create_tenant plugin for grafana-7.5.17-1-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:03.731] [DEBUG] - No such create_tenant plugin for grafana-7.5.17
[2025-04-29 11:22:03.731] [DEBUG] - Searching create_tenant plugin for prometheus-2.37.1-10000102022110211.el7-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:03.731] [DEBUG] - No such create_tenant plugin for prometheus-2.37.1
[2025-04-29 11:22:03.731] [DEBUG] - Searching tenant_optimize plugin for components …
[2025-04-29 11:22:03.731] [DEBUG] - Searching tenant_optimize plugin for obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.731] [DEBUG] - No such tenant_optimize plugin for obagent-4.2.2
[2025-04-29 11:22:03.731] [DEBUG] - Searching tenant_optimize plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:03.731] [DEBUG] - No such tenant_optimize plugin for obproxy-ce-4.2.3.0
[2025-04-29 11:22:03.731] [DEBUG] - Searching tenant_optimize plugin for oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:03.732] [DEBUG] - Found for oceanbase-ce-py_script_tenant_optimize-4.3.0.0 for oceanbase-ce-4.3.0.1
[2025-04-29 11:22:03.732] [DEBUG] - Searching tenant_optimize plugin for grafana-7.5.17-1-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:03.732] [DEBUG] - No such tenant_optimize plugin for grafana-7.5.17
[2025-04-29 11:22:03.732] [DEBUG] - Searching tenant_optimize plugin for prometheus-2.37.1-10000102022110211.el7-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:03.732] [DEBUG] - No such tenant_optimize plugin for prometheus-2.37.1
[2025-04-29 11:22:03.732] [DEBUG] - Searching start plugin for components …
[2025-04-29 11:22:03.732] [DEBUG] - Searching start plugin for obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.732] [DEBUG] - Found for obagent-py_script_start-1.3.0 for obagent-4.2.2
[2025-04-29 11:22:03.732] [DEBUG] - Searching start plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:03.733] [DEBUG] - Found for obproxy-ce-py_script_start-4.2.3 for obproxy-ce-4.2.3.0
[2025-04-29 11:22:03.733] [DEBUG] - Searching start plugin for oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:03.733] [DEBUG] - Found for oceanbase-ce-py_script_start-4.3.0.0 for oceanbase-ce-4.3.0.1
[2025-04-29 11:22:03.733] [DEBUG] - Searching start plugin for grafana-7.5.17-1-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:03.733] [DEBUG] - Found for grafana-py_script_start-7.5.17 for grafana-7.5.17
[2025-04-29 11:22:03.733] [DEBUG] - Searching start plugin for prometheus-2.37.1-10000102022110211.el7-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:03.733] [DEBUG] - Found for prometheus-py_script_start-2.37.1 for prometheus-2.37.1
[2025-04-29 11:22:03.734] [DEBUG] - Searching connect plugin for components …
[2025-04-29 11:22:03.734] [DEBUG] - Searching connect plugin for obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.734] [DEBUG] - Found for obagent-py_script_connect-1.3.0 for obagent-4.2.2
[2025-04-29 11:22:03.734] [DEBUG] - Searching connect plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:03.734] [DEBUG] - Found for obproxy-ce-py_script_connect-3.1.0 for obproxy-ce-4.2.3.0
[2025-04-29 11:22:03.734] [DEBUG] - Searching connect plugin for oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:03.734] [DEBUG] - Found for oceanbase-ce-py_script_connect-4.2.2.0 for oceanbase-ce-4.3.0.1
[2025-04-29 11:22:03.735] [DEBUG] - Searching connect plugin for grafana-7.5.17-1-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:03.735] [DEBUG] - Found for grafana-py_script_connect-7.5.17 for grafana-7.5.17
[2025-04-29 11:22:03.735] [DEBUG] - Searching connect plugin for prometheus-2.37.1-10000102022110211.el7-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:03.735] [DEBUG] - Found for prometheus-py_script_connect-2.37.1 for prometheus-2.37.1
[2025-04-29 11:22:03.735] [DEBUG] - Searching bootstrap plugin for components …
[2025-04-29 11:22:03.735] [DEBUG] - Searching bootstrap plugin for obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.735] [DEBUG] - Found for obagent-py_script_bootstrap-0.1 for obagent-4.2.2
[2025-04-29 11:22:03.735] [DEBUG] - Searching bootstrap plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:03.736] [DEBUG] - Found for obproxy-ce-py_script_bootstrap-3.1.0 for obproxy-ce-4.2.3.0
[2025-04-29 11:22:03.736] [DEBUG] - Searching bootstrap plugin for oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:03.736] [DEBUG] - Found for oceanbase-ce-py_script_bootstrap-4.2.2.0 for oceanbase-ce-4.3.0.1
[2025-04-29 11:22:03.736] [DEBUG] - Searching bootstrap plugin for grafana-7.5.17-1-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:03.736] [DEBUG] - Found for grafana-py_script_bootstrap-7.5.17 for grafana-7.5.17
[2025-04-29 11:22:03.736] [DEBUG] - Searching bootstrap plugin for prometheus-2.37.1-10000102022110211.el7-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:03.736] [DEBUG] - Found for prometheus-py_script_bootstrap-2.37.1 for prometheus-2.37.1
[2025-04-29 11:22:03.736] [DEBUG] - Searching display plugin for components …
[2025-04-29 11:22:03.736] [DEBUG] - Searching display plugin for obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.737] [DEBUG] - Found for obagent-py_script_display-1.3.0 for obagent-4.2.2
[2025-04-29 11:22:03.737] [DEBUG] - Searching display plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:03.737] [DEBUG] - Found for obproxy-ce-py_script_display-3.1.0 for obproxy-ce-4.2.3.0
[2025-04-29 11:22:03.737] [DEBUG] - Searching display plugin for oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:03.737] [DEBUG] - Found for oceanbase-ce-py_script_display-3.1.0 for oceanbase-ce-4.3.0.1
[2025-04-29 11:22:03.737] [DEBUG] - Searching display plugin for grafana-7.5.17-1-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:03.737] [DEBUG] - Found for grafana-py_script_display-7.5.17 for grafana-7.5.17
[2025-04-29 11:22:03.737] [DEBUG] - Searching display plugin for prometheus-2.37.1-10000102022110211.el7-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:03.738] [DEBUG] - Found for prometheus-py_script_display-2.37.1 for prometheus-2.37.1
[2025-04-29 11:22:03.858] [INFO] Load cluster param plugin
[2025-04-29 11:22:03.859] [DEBUG] - Get local repository obagent-4.2.2-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.859] [DEBUG] - Get local repository obproxy-ce-4.2.3.0-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:03.859] [DEBUG] - Get local repository oceanbase-ce-4.3.0.1-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:03.859] [DEBUG] - Get local repository grafana-7.5.17-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:03.859] [DEBUG] - Get local repository prometheus-2.37.1-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:03.859] [DEBUG] - Searching param plugin for components …
[2025-04-29 11:22:03.860] [DEBUG] - Search param plugin for obagent
[2025-04-29 11:22:03.860] [DEBUG] - Found for obagent-param-1.3.0 for obagent-4.2.2
[2025-04-29 11:22:03.860] [DEBUG] - Applying obagent-param-1.3.0 for obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:03.927] [DEBUG] - Search param plugin for obproxy-ce
[2025-04-29 11:22:03.927] [DEBUG] - Found for obproxy-ce-param-4.2.3 for obproxy-ce-4.2.3.0
[2025-04-29 11:22:03.927] [DEBUG] - Applying obproxy-ce-param-4.2.3 for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:04.047] [DEBUG] - Search param plugin for oceanbase-ce
[2025-04-29 11:22:04.048] [DEBUG] - Found for oceanbase-ce-param-4.3.0.0 for oceanbase-ce-4.3.0.1
[2025-04-29 11:22:04.048] [DEBUG] - Applying oceanbase-ce-param-4.3.0.0 for oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:04.540] [DEBUG] - Search param plugin for grafana
[2025-04-29 11:22:04.541] [DEBUG] - Found for grafana-param-7.5.17 for grafana-7.5.17
[2025-04-29 11:22:04.541] [DEBUG] - Applying grafana-param-7.5.17 for grafana-7.5.17-1-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:04.564] [DEBUG] - Search param plugin for prometheus
[2025-04-29 11:22:04.565] [DEBUG] - Found for prometheus-param-2.37.1 for prometheus-2.37.1
[2025-04-29 11:22:04.565] [DEBUG] - Applying prometheus-param-2.37.1 for prometheus-2.37.1-10000102022110211.el7-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:04.695] [INFO] Open ssh connection
[2025-04-29 11:22:04.827] [DEBUG] - Call oceanbase-ce-py_script_start_check-4.2.2.0 for oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:04.827] [DEBUG] - import start_check
[2025-04-29 11:22:04.834] [DEBUG] - add start_check ref count to 1
[2025-04-29 11:22:04.834] [INFO] Check before start observer
[2025-04-29 11:22:04.836] [DEBUG] – local execute: ls /root/oceanbase-ce/store/clog/tenant_1/
[2025-04-29 11:22:04.841] [DEBUG] – exited code 0
[2025-04-29 11:22:04.842] [DEBUG] – local execute: cat /root/oceanbase-ce/run/observer.pid
[2025-04-29 11:22:04.845] [DEBUG] – exited code 0
[2025-04-29 11:22:04.845] [DEBUG] – local execute: ls /proc/3496
[2025-04-29 11:22:04.850] [DEBUG] – exited code 2, error output:
[2025-04-29 11:22:04.850] [DEBUG] ls: 无法访问/proc/3496: 没有那个文件或目录
[2025-04-29 11:22:04.850] [DEBUG]
[2025-04-29 11:22:04.850] [DEBUG] – 127.0.0.1 port check
[2025-04-29 11:22:04.850] [DEBUG] – local execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{if($4==“0A”) print $2,$4,$10}’ | grep ‘:0B41’ | awk -F’ ’ ‘{print $3}’ | uniq
[2025-04-29 11:22:04.857] [DEBUG] – exited code 0
[2025-04-29 11:22:04.857] [DEBUG] – local execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{if($4==“0A”) print $2,$4,$10}’ | grep ‘:0B42’ | awk -F’ ’ ‘{print $3}’ | uniq
[2025-04-29 11:22:04.865] [DEBUG] – exited code 0
[2025-04-29 11:22:04.865] [DEBUG] – local execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{if($4==“0A”) print $2,$4,$10}’ | grep ‘:0B46’ | awk -F’ ’ ‘{print $3}’ | uniq
[2025-04-29 11:22:04.872] [DEBUG] – exited code 0
[2025-04-29 11:22:04.872] [DEBUG] – local execute: ls /root/oceanbase-ce/store/sstable/block_file
[2025-04-29 11:22:04.877] [DEBUG] – exited code 0
[2025-04-29 11:22:04.877] [DEBUG] – local execute: [ -w /tmp/ ] || [ -w /tmp/obshell ]
[2025-04-29 11:22:04.880] [DEBUG] – exited code 0
[2025-04-29 11:22:04.880] [DEBUG] – local execute: cat /proc/sys/fs/aio-max-nr /proc/sys/fs/aio-nr
[2025-04-29 11:22:04.883] [DEBUG] – exited code 0
[2025-04-29 11:22:04.883] [DEBUG] – local execute: ulimit -a
[2025-04-29 11:22:04.886] [DEBUG] – exited code 0
[2025-04-29 11:22:04.887] [WARNING] OBD-1007: (127.0.0.1) The recommended number of max user processes is 655350 (Current value: 120000)
[2025-04-29 11:22:04.887] [DEBUG] – local execute: sysctl -a
[2025-04-29 11:22:04.922] [DEBUG] – exited code 0
[2025-04-29 11:22:04.926] [WARNING] OBD-1017: (127.0.0.1) The value of the “vm.max_map_count” must be within [327600, 1310720] (Current value: 65530, Recommended value: 655360)
[2025-04-29 11:22:04.926] [DEBUG] – local execute: cat /proc/meminfo
[2025-04-29 11:22:04.930] [DEBUG] – exited code 0
[2025-04-29 11:22:04.930] [DEBUG] – local execute: df --block-size=1024
[2025-04-29 11:22:04.934] [DEBUG] – exited code 0
[2025-04-29 11:22:04.935] [DEBUG] – get disk info for path /dev, total: 13069365248 avail: 13069365248
[2025-04-29 11:22:04.935] [DEBUG] – get disk info for path /dev/shm, total: 13081649152 avail: 13081649152
[2025-04-29 11:22:04.935] [DEBUG] – get disk info for path /run, total: 13081649152 avail: 13047009280
[2025-04-29 11:22:04.935] [DEBUG] – get disk info for path /sys/fs/cgroup, total: 13081649152 avail: 13081649152
[2025-04-29 11:22:04.935] [DEBUG] – get disk info for path /, total: 992102842368 avail: 528803721216
[2025-04-29 11:22:04.935] [DEBUG] – get disk info for path /boot, total: 1063256064 avail: 905920512
[2025-04-29 11:22:04.935] [DEBUG] – get disk info for path /home, total: 90146082816 avail: 90112217088
[2025-04-29 11:22:04.935] [DEBUG] – get disk info for path /run/user/0, total: 2616332288 avail: 2616332288
[2025-04-29 11:22:04.935] [DEBUG] – get disk info for path /run/user/1003, total: 2616332288 avail: 2616332288
[2025-04-29 11:22:04.935] [DEBUG] – disk: {’/dev’: {‘total’: 13069365248, ‘avail’: 13069365248, ‘need’: 0}, ‘/dev/shm’: {‘total’: 13081649152, ‘avail’: 13081649152, ‘need’: 0}, ‘/run’: {‘total’: 13081649152, ‘avail’: 13047009280, ‘need’: 0}, ‘/sys/fs/cgroup’: {‘total’: 13081649152, ‘avail’: 13081649152, ‘need’: 0}, ‘/’: {‘total’: 992102842368, ‘avail’: 528803721216, ‘need’: 0}, ‘/boot’: {‘total’: 1063256064, ‘avail’: 905920512, ‘need’: 0}, ‘/home’: {‘total’: 90146082816, ‘avail’: 90112217088, ‘need’: 0}, ‘/run/user/0’: {‘total’: 2616332288, ‘avail’: 2616332288, ‘need’: 0}, ‘/run/user/1003’: {‘total’: 2616332288, ‘avail’: 2616332288, ‘need’: 0}}
[2025-04-29 11:22:04.936] [DEBUG] – local execute: date +%s%N
[2025-04-29 11:22:04.939] [DEBUG] – exited code 0
[2025-04-29 11:22:04.939] [DEBUG] – 127.0.0.1 time delta -0.41455078125
[2025-04-29 11:22:04.967] [INFO] [WARN] OBD-1007: (127.0.0.1) The recommended number of max user processes is 655350 (Current value: 120000)
[2025-04-29 11:22:04.967] [INFO] [WARN] OBD-1017: (127.0.0.1) The value of the “vm.max_map_count” must be within [327600, 1310720] (Current value: 65530, Recommended value: 655360)
[2025-04-29 11:22:04.967] [INFO]
[2025-04-29 11:22:04.967] [DEBUG] - sub start_check ref count to 0
[2025-04-29 11:22:04.967] [DEBUG] - export start_check
[2025-04-29 11:22:04.967] [DEBUG] - Call obagent-py_script_start_check-1.3.0 for obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e
[2025-04-29 11:22:04.968] [DEBUG] - import start_check
[2025-04-29 11:22:04.970] [DEBUG] - add start_check ref count to 1
[2025-04-29 11:22:04.970] [INFO] Check before start obagent
[2025-04-29 11:22:04.972] [DEBUG] – local execute: cat /root/obagent/run/ob_agentd.pid
[2025-04-29 11:22:04.976] [DEBUG] – exited code 1, error output:
[2025-04-29 11:22:04.976] [DEBUG] cat: /root/obagent/run/ob_agentd.pid: 没有那个文件或目录
[2025-04-29 11:22:04.976] [DEBUG]
[2025-04-29 11:22:04.976] [DEBUG] – 127.0.0.1 port check
[2025-04-29 11:22:04.976] [DEBUG] – local execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{if($4==“0A”) print $2,$4,$10}’ | grep ‘:1F99’ | awk -F’ ’ ‘{print $3}’ | uniq
[2025-04-29 11:22:04.983] [DEBUG] – exited code 0
[2025-04-29 11:22:04.983] [DEBUG] – local execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{if($4==“0A”) print $2,$4,$10}’ | grep ‘:1F98’ | awk -F’ ’ ‘{print $3}’ | uniq
[2025-04-29 11:22:04.990] [DEBUG] – exited code 0
[2025-04-29 11:22:05.103] [DEBUG] - sub start_check ref count to 0
[2025-04-29 11:22:05.103] [DEBUG] - export start_check
[2025-04-29 11:22:05.103] [DEBUG] - Call obproxy-ce-py_script_start_check-4.2.3 for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639
[2025-04-29 11:22:05.103] [DEBUG] - import start_check
[2025-04-29 11:22:05.106] [DEBUG] - add start_check ref count to 1
[2025-04-29 11:22:05.107] [INFO] Check before start obproxy
[2025-04-29 11:22:05.108] [DEBUG] – local execute: cat /root/obproxy-ce/run/obproxy-127.0.0.1-2883.pid
[2025-04-29 11:22:05.113] [DEBUG] – exited code 0
[2025-04-29 11:22:05.113] [DEBUG] – local execute: ls /proc/11868/fd
[2025-04-29 11:22:05.118] [DEBUG] – exited code 2, error output:
[2025-04-29 11:22:05.119] [DEBUG] ls: 无法访问/proc/11868/fd: 没有那个文件或目录
[2025-04-29 11:22:05.119] [DEBUG]
[2025-04-29 11:22:05.119] [DEBUG] – 127.0.0.1 port check
[2025-04-29 11:22:05.119] [DEBUG] – local execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{if($4==“0A”) print $2,$4,$10}’ | grep ‘:0B43’ | awk -F’ ’ ‘{print $3}’ | uniq
[2025-04-29 11:22:05.127] [DEBUG] – exited code 0
[2025-04-29 11:22:05.127] [DEBUG] – local execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{if($4==“0A”) print $2,$4,$10}’ | grep ‘:0B44’ | awk -F’ ’ ‘{print $3}’ | uniq
[2025-04-29 11:22:05.135] [DEBUG] – exited code 0
[2025-04-29 11:22:05.239] [DEBUG] - sub start_check ref count to 0
[2025-04-29 11:22:05.239] [DEBUG] - export start_check
[2025-04-29 11:22:05.239] [DEBUG] - Call prometheus-py_script_start_check-2.37.1 for prometheus-2.37.1-10000102022110211.el7-58913c7606f05feb01bc1c6410346e5fc31cf263
[2025-04-29 11:22:05.239] [DEBUG] - import start_check
[2025-04-29 11:22:05.241] [DEBUG] - add start_check ref count to 1
[2025-04-29 11:22:05.242] [INFO] Check before start prometheus
[2025-04-29 11:22:05.243] [DEBUG] – local execute: cat /root/prometheus/run/prometheus.pid
[2025-04-29 11:22:05.247] [DEBUG] – exited code 0
[2025-04-29 11:22:05.247] [DEBUG] – 127.0.0.1 port check
[2025-04-29 11:22:05.247] [DEBUG] – local execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{if($4==“0A”) print $2,$4,$10}’ | grep ‘:2382’ | awk -F’ ’ ‘{print $3}’ | uniq
[2025-04-29 11:22:05.255] [DEBUG] – exited code 0
[2025-04-29 11:22:05.374] [DEBUG] - sub start_check ref count to 0
[2025-04-29 11:22:05.374] [DEBUG] - export start_check
[2025-04-29 11:22:05.374] [DEBUG] - Call grafana-py_script_start_check-7.5.17 for grafana-7.5.17-1-1bf1f338d3a3445d8599dc6902e7aeed4de4e0d6
[2025-04-29 11:22:05.374] [DEBUG] - import start_check
[2025-04-29 11:22:05.376] [DEBUG] - add start_check ref count to 1
[2025-04-29 11:22:05.377] [INFO] Check before start grafana
[2025-04-29 11:22:05.377] [DEBUG] – local execute: cat /root/grafana/run/grafana.pid
[2025-04-29 11:22:05.381] [DEBUG] – exited code 0
[2025-04-29 11:22:05.382] [DEBUG] – local execute: ls /proc/12002
[2025-04-29 11:22:05.386] [DEBUG] – exited code 2, error output:
[2025-04-29 11:22:05.387] [DEBUG] ls: 无法访问/proc/12002: 没有那个文件或目录
[2025-04-29 11:22:05.387] [DEBUG]
[2025-04-29 11:22:05.387] [DEBUG] – 127.0.0.1 port check
[2025-04-29 11:22:05.387] [DEBUG] – local execute: bash -c ‘cat /proc/net/{udp*,tcp*}’ | awk -F’ ’ ‘{if($4==“0A”) print $2,$4,$10}’ | grep ‘:0BB8’ | awk -F’ ’ ‘{print $3}’ | uniq
[2025-04-29 11:22:05.394] [DEBUG] – exited code 0
[2025-04-29 11:22:05.508] [DEBUG] - sub start_check ref count to 0
[2025-04-29 11:22:05.508] [DEBUG] - export start_check
[2025-04-29 11:22:05.509] [DEBUG] - Call oceanbase-ce-py_script_start-4.3.0.0 for oceanbase-ce-4.3.0.1-100000242024032211.el7-c4a03c83614f50c99ddb1c37dda858fa5d9b14b7
[2025-04-29 11:22:05.509] [DEBUG] - import start
[2025-04-29 11:22:05.512] [DEBUG] - add start ref count to 1
[2025-04-29 11:22:05.513] [INFO] Start observer
[2025-04-29 11:22:05.514] [DEBUG] – local execute: ls /root/oceanbase-ce/store/clog/tenant_1/
[2025-04-29 11:22:05.519] [DEBUG] – exited code 0
[2025-04-29 11:22:05.519] [DEBUG] – local execute: cat /root/oceanbase-ce/run/observer.pid
[2025-04-29 11:22:05.522] [DEBUG] – exited code 0
[2025-04-29 11:22:05.523] [DEBUG] – local execute: ls /proc/3496
[2025-04-29 11:22:05.527] [DEBUG] – exited code 2, error output:
[2025-04-29 11:22:05.527] [DEBUG] ls: 无法访问/proc/3496: 没有那个文件或目录
[2025-04-29 11:22:05.527] [DEBUG]
[2025-04-29 11:22:05.527] [DEBUG] – 127.0.0.1 start command construction
[2025-04-29 11:22:05.528] [DEBUG] – local execute: ls /root/oceanbase-ce/etc/observer.config.bin
[2025-04-29 11:22:05.532] [DEBUG] – exited code 0
[2025-04-29 11:22:05.532] [DEBUG] – starting 127.0.0.1 observer
[2025-04-29 11:22:05.533] [DEBUG] – root@127.0.0.1 set env LD_LIBRARY_PATH to ‘/root/oceanbase-ce/lib:’
[2025-04-29 11:22:05.533] [DEBUG] – local execute: cd /root/oceanbase-ce; /root/oceanbase-ce/bin/observer -p 2881
[2025-04-29 11:22:05.580] [DEBUG] – exited code 0
[2025-04-29 11:22:05.581] [DEBUG] – root@127.0.0.1 delete env LD_LIBRARY_PATH
[2025-04-29 11:22:05.645] [DEBUG] – start_obshell: True
[2025-04-29 11:22:05.645] [DEBUG] – local execute: cat /root/oceanbase-ce/run/obshell.pid
[2025-04-29 11:22:05.649] [DEBUG] – exited code 0
[2025-04-29 11:22:05.650] [DEBUG] – local execute: ls /proc/3565
[2025-04-29 11:22:05.655] [DEBUG] – exited code 2, error output:
[2025-04-29 11:22:05.655] [DEBUG] ls: 无法访问/proc/3565: 没有那个文件或目录
[2025-04-29 11:22:05.655] [DEBUG]
[2025-04-29 11:22:05.655] [DEBUG] – root@127.0.0.1 set env OB_ROOT_PASSWORD to ‘’
[2025-04-29 11:22:05.655] [DEBUG] – start obshell: cd /root/oceanbase-ce; /root/oceanbase-ce/bin/obshell admin start --ip 127.0.0.1 --port 2886
[2025-04-29 11:22:05.655] [DEBUG] – local execute: cd /root/oceanbase-ce; /root/oceanbase-ce/bin/obshell admin start --ip 127.0.0.1 --port 2886
[2025-04-29 11:22:07.714] [DEBUG] – exited code 0
[2025-04-29 11:22:07.714] [INFO] observer program health check
[2025-04-29 11:22:10.718] [DEBUG] – 127.0.0.1 program health check
[2025-04-29 11:22:10.718] [DEBUG] – local execute: cat /root/oceanbase-ce/run/observer.pid
[2025-04-29 11:22:10.723] [DEBUG] – exited code 0
[2025-04-29 11:22:10.723] [DEBUG] – local execute: ls /proc/14248
[2025-04-29 11:22:10.728] [DEBUG] – exited code 2, error output:
[2025-04-29 11:22:10.728] [DEBUG] ls: 无法访问/proc/14248: 没有那个文件或目录
[2025-04-29 11:22:10.728] [DEBUG]
[2025-04-29 11:22:10.842] [WARNING] OBD-2002: Failed to start 127.0.0.1 observer
[2025-04-29 11:22:10.842] [DEBUG] - sub start ref count to 0
[2025-04-29 11:22:10.843] [DEBUG] - export start
[2025-04-29 11:22:10.843] [ERROR] oceanbase-ce start failed
[2025-04-29 11:22:10.846] [INFO] See https://www.oceanbase.com/product/ob-deployer/error-codes .
[2025-04-29 11:22:10.846] [INFO] Trace ID: 1ef13938-24a9-11f0-a7c8-005056b7a1e2
[2025-04-29 11:22:10.846] [INFO] If you want to view detailed obd logs, please run: obd display-trace 1ef13938-24a9-11f0-a7c8-005056b7a1e2
[2025-04-29 11:22:10.846] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 4
[2025-04-29 11:22:10.846] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 3
[2025-04-29 11:22:10.846] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 2
[2025-04-29 11:22:10.846] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 1
[2025-04-29 11:22:10.846] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 0
[2025-04-29 11:22:10.846] [DEBUG] - unlock /root/.obd/lock/mirror_and_repo
[2025-04-29 11:22:10.847] [DEBUG] - exclusive lock /root/.obd/lock/deploy_demo release, count 0
[2025-04-29 11:22:10.847] [DEBUG] - unlock /root/.obd/lock/deploy_demo
[2025-04-29 11:22:10.847] [DEBUG] - share lock /root/.obd/lock/global release, count 0
[2025-04-29 11:22:10.847] [DEBUG] - unlock /root/.obd/lock/global

2 个赞
#11

我已经KILL掉了，那个是启动到一半就会报，请看我发的最新启动日志，谢谢

2 个赞
#12

根据您提供的日志和问题描述，以下是可能的原因分析及解决方法：

1. 端口冲突

  • 日志中显示 OBD-1001: 127.0.0.1:2886 port is already used，表明 2886 端口已被占用。
  • 解决方法：
    • 检查是否有其他进程占用了该端口：
      netstat -tulnp | grep 2886
    • 如果有其他进程占用，请终止该进程或更改 OceanBase 的监听端口。可以通过以下命令编辑配置文件并修改端口：
      obd cluster edit-config demo
      修改 global 部分的 rpc_port 和 sql_port 参数，例如：
      global:
  rpc_port: 2887
  sql_port: 2888

2. 系统参数不足

  • 日志中显示两个警告信息：
    • OBD-1007: The recommended number of max user processes is 655350 (Current value: 120000)
    • OBD-1017: The value of the "vm.max_map_count" must be within [327600, 1310720] (Current value: 65530)
  • 解决方法：
    • 调整系统参数以满足推荐值：
      # 修改最大用户进程数
echo "* soft nofile 655350" >> /etc/security/limits.conf
echo "* hard nofile 655350" >> /etc/security/limits.conf

# 修改 vm.max_map_count
sysctl -w vm.max_map_count=655360
echo "vm.max_map_count=655360" >> /etc/sysctl.conf
    • 重启系统以使更改生效。

3. observer 进程未正常启动

  • 日志中多次出现类似以下错误：
    ls: 无法访问/proc/3496: 没有那个文件或目录
    表明 observer 进程启动后未能持续运行。
  • 解决方法：
    • 检查 observer 启动日志以获取更多详细信息：
      cat /root/oceanbase-ce/log/observer.log
    • 如果发现内存不足或资源限制导致进程退出，请调整相关参数（如 memory_limit 和 system_memory）：
      global:
  memory_limit: 8192M
  system_memory: 2048M

4. obshell 启动失败

  • 日志中显示 obshell 启动失败：
    start obshell: cd /root/oceanbase-ce; /root/oceanbase-ce/bin/obshell admin start --ip 127.0.0.1 --port 2886
exited code 0
    但后续健康检查失败：
    OBD-2002: Failed to start 127.0.0.1 observer
  • 解决方法：
    • 检查 obshell 的日志文件以定位具体问题：
      cat /root/oceanbase-ce/log/obshell.log
    • 如果是网络问题，请确保本地网络配置正确，并尝试重新启动 obshell。

5. 验证集群状态

  • 确保集群的配置文件正确无误，并且所有依赖组件均已正确部署。
  • 解决方法：
    • 使用以下命令检查集群状态：
      obd cluster status demo
    • 如果发现某些组件未正常运行，可以尝试重新部署或修复集群：
      obd cluster redeploy demo

总结

根据上述分析，请按照以下步骤逐步排查问题：

  1. 检查并释放占用的 2886 端口，或者修改 OceanBase 的监听端口。
  2. 调整系统参数（如 max user processes 和 vm.max_map_count），以满足推荐值。
  3. 检查 observer 和 obshell 的日志文件，定位具体失败原因。
  4. 验证集群状态，必要时重新部署或修复集群。

如果以上方法仍无法解决问题，请提供更详细的日志信息，或者联系 OceanBase 的人工答疑以获取进一步支持。

详情请参考:

  1. 备份目录结构

(小助手的答复已结束，如未能解决您的问题，请继续提问并等待其他同学的回复，谢谢！)

2 个赞
#13

麻烦提供一份observer日志吧

1 个赞
#14

obdtoady.zip (12.0 KB)

2 个赞
#15

observer日志在demo集群的home目录下
~/demo/oceanbase/log下的observer日志

2 个赞
#16

稍等，我刚重启了，正在启动，大概率后面是失败的，我试了很多次了，失败后我拿日志给你

图片
图片1353×379 11.6 KB

1 个赞
#17

请问是哪个，我没有demo目录

图片
图片779×520 12.1 KB

2 个赞
#18

找到了，请查阅
observer.log (3).zip (14.9 MB)

2 个赞
#19

log_disk_base_path="/root/oceanbase-ce/store/clog"
这个路径的文件是否动过，报错clog问题。
麻烦铲掉demo集群，升级一下OBD，重建一个新版本435或425集群试试

1 个赞
#20

没动过这个路径
log_disk_base_path="/root/oceanbase-ce/store/clog"
这个在哪里看？3月28我们断过一次电，但是后面我启动时成功的。

麻烦铲掉demo集群，升级一下OBD，重建一个新版本435或425集群试试，这个以前的数据还有租户库表都还在吗？

#21

那可能是断电导致log文件损坏了
demo集群是测试集群，不建议进行生产部署使用的。

image
image1210×246 85.1 KB

#22

我们用在开发环境，因为我们有客户用的你们产品，所以要适配，那这个要怎么解决呢？或者有没有办法把里面数据库表信息等拿出来，我重建也可以