[WARN] OBD-2002: Failed to start 172.17.0.2 observer

【 使用环境 】测试环境
【 OB or 其他组件 】OBD、oceanbase-ce
【 使用版本 】OBD 2.9.2、oceanbase-ce 4.2.1.8
【问题描述】使用默认安装配置,其他步骤都执行成功,只有 observer 启动这一步一直失败。
【复现路径】参考 https://open.oceanbase.com/quickStart 方案三,使用 Docker 镜像安装后,observer 启动失败:

【附件及日志】推荐使用OceanBase敏捷诊断工具obdiag收集诊断信息,详情参见链接(右键跳转查看):

【SOP系列 22 】——故障诊断第一步(自助诊断和诊断信息收集)

2024-09-06 16:44:47 +--------------------------------------------------+
2024-09-06 16:44:47 |                   Cluster List                   |
2024-09-06 16:44:47 +------+-------------------------+-----------------+
2024-09-06 16:44:47 | Name | Configuration Path      | Status (Cached) |
2024-09-06 16:44:47 +------+-------------------------+-----------------+
2024-09-06 16:44:47 | demo | /root/.obd/cluster/demo | stopped         |
2024-09-06 16:44:47 +------+-------------------------+-----------------+
2024-09-06 16:44:47 Trace ID: 45b1d9b4-6c2c-11ef-a65e-0242ac110002
2024-09-06 16:44:47 If you want to view detailed obd logs, please run: obd display-trace 45b1d9b4-6c2c-11ef-a65e-0242ac110002
2024-09-06 16:44:47 repository/
2024-09-06 16:44:47 repository/oceanbase-ce-libs/
2024-09-06 16:44:47 repository/oceanbase-ce-libs/4.2.1.8/
2024-09-06 16:44:47 repository/oceanbase-ce-libs/4.2.1.8/d02f4bfd321370a02550424293beb1be31204038/
2024-09-06 16:44:47 repository/oceanbase-ce-libs/4.2.1.8/d02f4bfd321370a02550424293beb1be31204038/libmariadb.so.3
2024-09-06 16:44:47 repository/oceanbase-ce-libs/4.2.1.8/d02f4bfd321370a02550424293beb1be31204038/.data
2024-09-06 16:44:47 repository/oceanbase-ce-libs/4.2.1.8/d02f4bfd321370a02550424293beb1be31204038/libaio.so
2024-09-06 16:44:47 repository/oceanbase-ce-libs/4.2.1.8/d02f4bfd321370a02550424293beb1be31204038/libaio.so.1
2024-09-06 16:44:47 repository/oceanbase-ce-libs/4.2.1.8/d02f4bfd321370a02550424293beb1be31204038/libaio.so.1.0.1
2024-09-06 16:44:47 repository/oceanbase-ce-libs/4.2.1.8/d02f4bfd321370a02550424293beb1be31204038/libmariadb.so
2024-09-06 16:44:47 repository/oceanbase-ce-libs/4.2.1.8/oceanbase-ce-libs
2024-09-06 16:44:47 repository/obagent/
2024-09-06 16:44:47 repository/obagent/4.2.2/
2024-09-06 16:44:47 repository/obagent/4.2.2/obagent
2024-09-06 16:44:47 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/
2024-09-06 16:44:47 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/.data
2024-09-06 16:44:47 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/bin/
2024-09-06 16:44:47 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/bin/ob_monagent
2024-09-06 16:44:47 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/bin/ob_mgragent
2024-09-06 16:44:47 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/bin/ob_agentd
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/bin/ob_agentctl
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/prometheus_config/
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/prometheus_config/rules/
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/prometheus_config/rules/ob_rules.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/prometheus_config/rules/host_rules.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/prometheus_config/prometheus.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/monagent.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/monitor_mysql.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/monitor_node_host.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/monitor_observer_log.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/log_module.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/monitor_host_log.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/mgragent_module.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/monitor_ob_custom.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/monitor_ob.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/common_module.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/monagent_basic_auth.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/monitor_ob_log.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/ob_logcleaner_module.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/module_config/mgragent_logquerier_module.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/agentctl.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/agentd.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/shell_templates/
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/shell_templates/shell_template.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/obd_agent_mapper.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/config_properties/
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/config_properties/basic_auth.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/config_properties/common_meta.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/config_properties/log.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/config_properties/ob_logcleaner.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/config_properties/monagent_pipeline.yaml
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/.config_secret.key
2024-09-06 16:44:48 repository/obagent/4.2.2/19739a07a12eab736aff86ecf357b1ae660b554e/conf/mgragent.yaml
2024-09-06 16:44:48 repository/ob-configserver/
2024-09-06 16:44:48 repository/ob-configserver/1.0.0/
2024-09-06 16:44:48 repository/ob-configserver/1.0.0/feca6b9c76e26ac49464f34bfa0780b5a8d3f4a0/
2024-09-06 16:44:48 repository/ob-configserver/1.0.0/feca6b9c76e26ac49464f34bfa0780b5a8d3f4a0/.data
2024-09-06 16:44:48 repository/ob-configserver/1.0.0/feca6b9c76e26ac49464f34bfa0780b5a8d3f4a0/bin/
2024-09-06 16:44:48 repository/ob-configserver/1.0.0/feca6b9c76e26ac49464f34bfa0780b5a8d3f4a0/bin/ob-configserver
2024-09-06 16:44:48 repository/ob-configserver/1.0.0/ob-configserver
2024-09-06 16:44:48 repository/oceanbase-ce/
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/.data
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_stats_body_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_stats_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_resource_manager_body_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_resource_manager_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_workload_repository_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_xplan_mysql_body.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_udr_body_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/__dbms_upgrade_body_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/json_object_type_body.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_application_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/json_element_type.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_scheduler_mysql_body.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/__dbms_upgrade_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_workload_repository_body_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_udr_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_workload_repository.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_monitor_body_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/json_object_type.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_monitor_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_ash_internal.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_application_body_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_rls_body.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_session_body_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/json_element_type_body.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_session_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_workload_repository_body.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_udr_body.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_scheduler_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_rls.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_udr.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_ash_internal_body.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/admin/dbms_xplan_mysql.sql
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/bin/
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/bin/obshell
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/bin/import_srs_data.py
2024-09-06 16:44:48 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/bin/observer
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/bin/import_time_zone_info.py
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/etc/
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/etc/fill_help_tables-ob.sql
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/etc/upgrade_health_checker.py
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/etc/upgrade_post.py
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/etc/default_srs_data_mysql.sql
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/etc/timezone_V1.log
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/etc/oceanbase_upgrade_dep.yml
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/etc/upgrade_checker.py
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/499b676f2ede5a16e0c07b2b15991d1160d972e8/etc/upgrade_pre.py
2024-09-06 16:44:51 repository/oceanbase-ce/4.2.1.8/oceanbase-ce
2024-09-06 16:44:51 do normal boot
2024-09-06 16:44:52 oceanbase-ce-4.2.1.8 already installed.
2024-09-06 16:44:52 obagent-4.2.2 already installed.
2024-09-06 16:44:52 ob-configserver-1.0.0 already installed.
2024-09-06 16:44:52 +-----------------------------------------------------------------------------------------------+
2024-09-06 16:44:52 |                                            Packages                                           |
2024-09-06 16:44:52 +-----------------+---------+------------------------+------------------------------------------+
2024-09-06 16:44:52 | Repository      | Version | Release                | Md5                                      |
2024-09-06 16:44:52 +-----------------+---------+------------------------+------------------------------------------+
2024-09-06 16:44:52 | oceanbase-ce    | 4.2.1.8 | 108000022024072217.el7 | 499b676f2ede5a16e0c07b2b15991d1160d972e8 |
2024-09-06 16:44:52 | obagent         | 4.2.2   | 100000042024011120.el7 | 19739a07a12eab736aff86ecf357b1ae660b554e |
2024-09-06 16:44:52 | ob-configserver | 1.0.0   | 2.el7                  | feca6b9c76e26ac49464f34bfa0780b5a8d3f4a0 |
2024-09-06 16:44:52 +-----------------+---------+------------------------+------------------------------------------+
2024-09-06 16:44:52 Repository integrity check ok
2024-09-06 16:44:52 Load param plugin ok
2024-09-06 16:44:53 Open ssh connection ok
2024-09-06 16:44:53 Parameter check ok
2024-09-06 16:44:53 Cluster status check ok
2024-09-06 16:44:54 Initializes observer work home ok
2024-09-06 16:44:54 Initializes obagent work home ok
2024-09-06 16:44:54 Initializes ob-configserver work home ok
2024-09-06 16:44:55 Remote oceanbase-ce-4.2.1.8-108000022024072217.el7-499b676f2ede5a16e0c07b2b15991d1160d972e8 repository install ok
2024-09-06 16:44:55 Remote oceanbase-ce-4.2.1.8-108000022024072217.el7-499b676f2ede5a16e0c07b2b15991d1160d972e8 repository lib check !!
2024-09-06 16:44:56 Remote obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e repository install ok
2024-09-06 16:44:56 Remote obagent-4.2.2-100000042024011120.el7-19739a07a12eab736aff86ecf357b1ae660b554e repository lib check ok
2024-09-06 16:44:57 Remote ob-configserver-1.0.0-2.el7-feca6b9c76e26ac49464f34bfa0780b5a8d3f4a0 repository install ok
2024-09-06 16:44:57 Remote ob-configserver-1.0.0-2.el7-feca6b9c76e26ac49464f34bfa0780b5a8d3f4a0 repository lib check ok
2024-09-06 16:44:57 Try to get lib-repository
2024-09-06 16:44:57 oceanbase-ce-libs-4.2.1.8 already installed.
2024-09-06 16:44:57 Remote oceanbase-ce-libs-4.2.1.8-108000022024072217.el7-d02f4bfd321370a02550424293beb1be31204038 repository install ok
2024-09-06 16:44:57 Remote oceanbase-ce-4.2.1.8-108000022024072217.el7-499b676f2ede5a16e0c07b2b15991d1160d972e8 repository lib check ok
2024-09-06 16:44:57 obcluster deployed
2024-09-06 16:44:57 Please execute ` obd cluster start obcluster ` to start
2024-09-06 16:44:57 Trace ID: 488386ce-6c2c-11ef-8c61-0242ac110002
2024-09-06 16:44:57 If you want to view detailed obd logs, please run: obd display-trace 488386ce-6c2c-11ef-8c61-0242ac110002
2024-09-06 16:44:58 Get local repositories ok
2024-09-06 16:44:58 Search plugins ok
2024-09-06 16:44:59 Load cluster param plugin ok
2024-09-06 16:44:59 Open ssh connection ok
2024-09-06 16:44:59 Check before start ob-configserver ok
2024-09-06 16:45:00 Check before start observer ok
2024-09-06 16:45:00 [WARN] OBD-1011: (172.17.0.2) The recommended value of fs.aio-max-nr is 1048576 (Current value: 65536)
2024-09-06 16:45:00 [WARN] OBD-1007: (172.17.0.2) The recommended number of core file size is unlimited (Current value: 0)
2024-09-06 16:45:00 [WARN] OBD-1007: (172.17.0.2) The recommended number of stack size is unlimited (Current value: 8192)
2024-09-06 16:45:00 [WARN] OBD-1017: (172.17.0.2) The value of the "vm.max_map_count" must be within [327600, 1310720] (Current value: 262144, Recommended value: 655360)
2024-09-06 16:45:00 [WARN] OBD-1017: (172.17.0.2) The value of the "vm.overcommit_memory" must be 0 (Current value: 1, Recommended value: 0)
2024-09-06 16:45:00 [WARN] OBD-1017: (172.17.0.2) The value of the "fs.file-max" must be greater than 6573688 (Current value: 1636382, Recommended value: 6573688)
2024-09-06 16:45:00 [WARN] OBD-1012: (172.17.0.2) clog and data use the same disk (/)
2024-09-06 16:45:00 
2024-09-06 16:45:00 Check before start obagent ok
2024-09-06 16:45:00 Start ob-configserver ok
2024-09-06 16:45:01 ob-configserver program health check ok
2024-09-06 16:45:01 Connect to ob-configserver ok
2024-09-06 16:45:02 Start observer ok
2024-09-06 16:45:05 observer program health check x
2024-09-06 16:45:05 [WARN] OBD-2002: Failed to start 172.17.0.2 observer
2024-09-06 16:45:05 [ERROR] oceanbase-ce start failed
2024-09-06 16:45:05 +------------------------------------------------------------+
2024-09-06 16:45:05 |                      ob-configserver                       |
2024-09-06 16:45:05 +------------+------+-------------+----------+--------+------+
2024-09-06 16:45:05 | server     | port | vip_address | vip_port | status | pid  |
2024-09-06 16:45:05 +------------+------+-------------+----------+--------+------+
2024-09-06 16:45:05 | 172.17.0.2 | 8080 | 172.17.0.2  | 8080     | active | 1201 |
2024-09-06 16:45:05 +------------+------+-------------+----------+--------+------+
2024-09-06 16:45:05 curl -s 'http://172.17.0.2:8080/services?Action=GetObProxyConfig'
2024-09-06 16:45:05 See https://www.oceanbase.com/product/ob-deployer/error-codes .
2024-09-06 16:45:05 Trace ID: 4c2307dc-6c2c-11ef-bbef-0242ac110002
2024-09-06 16:45:05 If you want to view detailed obd logs, please run: obd display-trace 4c2307dc-6c2c-11ef-bbef-0242ac110002
2024-09-06 16:45:05 


boot failed!

安装配置 (使用官方默认配置,没有自定义):

oceanbase-ce:
  depends:
  - ob-configserver
  servers:
  - 172.17.0.2
  global:
    home_path: /root/ob
    mysql_port: 2881
    rpc_port: 2882
    zone: zone1
    cluster_id: 1
    appname: obcluster
    memory_limit: 6G
    system_memory: 1G
    datafile_size: 5G
    log_disk_size: 5G
    root_password:
    scenario: express_oltp
    cpu_count: 16
    production_mode: false
    syslog_level: INFO
    enable_syslog_wf: false
    enable_syslog_recycle: true
    max_syslog_file_count: 4
    enable_rich_error_msg: true
    ocp_agent_monitor_password: Xzd40zc3kR
obagent:
  # The list of servers to be monitored. This list is consistent with the servers in oceanbase-ce.
  servers:
  - 172.17.0.2
  # Set dependent components for the component.
  # When the associated configurations are not done, OBD will automatically get the these configurations from the dependent components.
  depends:
  - oceanbase-ce
  global:
    # The working directory for obagent. obagent is started under this directory. This is a required field.
    home_path: /root/obagent
    # The port that pulls and manages the metrics. The default port number is 8088.
    monagent_http_port: 8088
    # Debug port for pprof. The default port number is 8089.
    mgragent_http_port: 8089
    # Log level. The default value is INFO.
    log_level: INFO
    # Log path. The default value is log/monagent.log.
    log_path: log/monagent.log
    # Encryption method. OBD supports aes and plain. The default value is plain.
    crypto_method: plain
    # Path to store the crypto key. The default value is conf/.config_secret.key.
    # crypto_path: conf/.config_secret.key
    # Size for a single log file. Log size is measured in Megabytes. The default value is 30M.
    log_size: 30
    # Expiration time for logs. The default value is 7 days.
    log_expire_day: 7
    # The maximum number for log files. The default value is 10.
    log_file_count: 10
    # Whether to use local time for log files. The default value is true.
    # log_use_localtime: true
    # Whether to enable log compression. The default value is true.
    # log_compress: true
    # Username for HTTP authentication. The default value is admin.
    http_basic_auth_user: admin
    # Password for HTTP authentication. The default value is root.
    http_basic_auth_password: root
    # Username for debug service. The default value is admin.
    pprof_basic_auth_user: admin
    # Password for debug service. The default value is root.
    pprof_basic_auth_password: root
    # Monitor username for OceanBase Database. The user must have read access to OceanBase Database as a system tenant. The default value is root.
    # monitor_user: root
    # Monitor password for OceanBase Database. The default value is empty. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the root_password in oceanbase-ce.
    # monitor_password:
    # The SQL port for observer. The default value is 2881. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the mysql_port in oceanbase-ce.
    # sql_port: 2881
    # The RPC port for observer. The default value is 2882. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the rpc_port in oceanbase-ce.
    # rpc_port: 2882
    # Cluster name for OceanBase Database. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the appname in oceanbase-ce.
    cluster_name: obcluster
    # Cluster ID for OceanBase Database. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the cluster_id in oceanbase-ce.
    # cluster_id: 1
    # Zone name for your observer. The default value is zone1. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the zone name in oceanbase-ce.
    # zone_name: zone1
    # Monitor status for OceanBase Database.  Active is to enable. Inactive is to disable. The default value is active. When you deploy an cluster automatically, OBD decides whether to enable this parameter based on depends.
    ob_monitor_status: active
    # Monitor status for your host. Active is to enable. Inactive is to disable. The default value is active.
    host_monitor_status: inactive
    # Whether to disable the basic authentication for HTTP service. True is to disable. False is to enable. The default value is false.
    disable_http_basic_auth: true
    # Whether to disable the basic authentication for the debug interface. True is to disable. False is to enable. The default value is false.
    disable_pprof_basic_auth: true
ob-configserver:
  servers:
  - 172.17.0.2
  global:
    listen_port: 8080
    server_ip: 0.0.0.0
    home_path: /root/obconfigserver

【备注】基于 LLM 和开源文档 RAG 的论坛小助手已开放测试,在发帖时输入 [@论坛小助手] 即可召唤小助手,欢迎试用!

2 个赞

根据文档库中的信息,您遇到的问题可能是由于操作系统的内核参数不在推荐的范围之内导致的 [1]。从日志信息中可以看出,存在多个内核参数不符合推荐值:

  1. fs.aio-max-nr 的当前值为 65536,推荐值为 1048576;
  2. 核心转储文件大小 (core file size) 当前限制为 0,推荐值为不限制;
  3. 栈大小 (stack size) 当前限制为 8192,推荐值为不限制;
  4. vm.max_map_count 的当前值为 262144,推荐值为 655360;
  5. vm.overcommit_memory 的当前值为 1,推荐值为 0;
  6. fs.file-max 的当前值为 1636382,推荐值为 6573688。

此外,还有一条警告提示 clog 和数据使用了同一块磁盘 ( [1])。

为了解决这个问题,您可以按照以下步骤操作:

  1. 修改内核参数以达到推荐值。可以通过执行 sysctl -w {内核参数名}="建议值" 命令或者 echo "内核参数名=建议值" >> /etc/sysctl.conf; sysctl -p 命令来修改参数配置。例如,要修改 fs.aio-max-nr 参数,您可以运行:

    sudo sysctl -w fs.aio-max-nr=1048576
    # 或
    echo "fs.aio-max-nr=1048576" >> /etc/sysctl.conf; sudo sysctl -p
    

    请针对每个不达标的参数重复上述操作。

  2. 如果您在测试环境中,并且没有权限修改内核参数,可以通过 obd cluster edit-config {deployname} 命令修改配置文件,将配置项 production_mode 设置为 false 来跳过系统参数阻塞检查 [1]

  3. 由于 clog 和数据使用了同一块磁盘,这可能会导致性能问题。虽然这不是阻止启动的直接原因,但建议您考虑将它们分配到不同的磁盘上以优化性能。

完成这些步骤后,再次尝试启动 observer 应该能够解决问题。如果问题依旧存在,请检查是否有其他错误信息,并考虑联系 OceanBase 的技术支持人员以获得进一步的帮助。

详情请参考:

  1. 错误码
  2. 4000 ~ 4499
  3. OceanBase Deployer(OBD)
2 个赞

该问题大部分是因为配置原因,可以贴一下你的yaml文件看一下集群配置参数

2 个赞

已在帖子中提供了安装配置,但这个是官方生成的默认配置,没有自定义过。

oceanbase-ce:
  depends:
  - ob-configserver
  servers:
  - 172.17.0.2
  global:
    home_path: /root/ob
    mysql_port: 2881
    rpc_port: 2882
    zone: zone1
    cluster_id: 1
    appname: obcluster
    memory_limit: 6G
    system_memory: 1G
    datafile_size: 5G
    log_disk_size: 5G
    root_password:
    scenario: express_oltp
    cpu_count: 16
    production_mode: false
    syslog_level: INFO
    enable_syslog_wf: false
    enable_syslog_recycle: true
    max_syslog_file_count: 4
    enable_rich_error_msg: true
    ocp_agent_monitor_password: Xzd40zc3kR
obagent:
  # The list of servers to be monitored. This list is consistent with the servers in oceanbase-ce.
  servers:
  - 172.17.0.2
  # Set dependent components for the component.
  # When the associated configurations are not done, OBD will automatically get the these configurations from the dependent components.
  depends:
  - oceanbase-ce
  global:
    # The working directory for obagent. obagent is started under this directory. This is a required field.
    home_path: /root/obagent
    # The port that pulls and manages the metrics. The default port number is 8088.
    monagent_http_port: 8088
    # Debug port for pprof. The default port number is 8089.
    mgragent_http_port: 8089
    # Log level. The default value is INFO.
    log_level: INFO
    # Log path. The default value is log/monagent.log.
    log_path: log/monagent.log
    # Encryption method. OBD supports aes and plain. The default value is plain.
    crypto_method: plain
    # Path to store the crypto key. The default value is conf/.config_secret.key.
    # crypto_path: conf/.config_secret.key
    # Size for a single log file. Log size is measured in Megabytes. The default value is 30M.
    log_size: 30
    # Expiration time for logs. The default value is 7 days.
    log_expire_day: 7
    # The maximum number for log files. The default value is 10.
    log_file_count: 10
    # Whether to use local time for log files. The default value is true.
    # log_use_localtime: true
    # Whether to enable log compression. The default value is true.
    # log_compress: true
    # Username for HTTP authentication. The default value is admin.
    http_basic_auth_user: admin
    # Password for HTTP authentication. The default value is root.
    http_basic_auth_password: root
    # Username for debug service. The default value is admin.
    pprof_basic_auth_user: admin
    # Password for debug service. The default value is root.
    pprof_basic_auth_password: root
    # Monitor username for OceanBase Database. The user must have read access to OceanBase Database as a system tenant. The default value is root.
    # monitor_user: root
    # Monitor password for OceanBase Database. The default value is empty. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the root_password in oceanbase-ce.
    # monitor_password:
    # The SQL port for observer. The default value is 2881. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the mysql_port in oceanbase-ce.
    # sql_port: 2881
    # The RPC port for observer. The default value is 2882. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the rpc_port in oceanbase-ce.
    # rpc_port: 2882
    # Cluster name for OceanBase Database. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the appname in oceanbase-ce.
    cluster_name: obcluster
    # Cluster ID for OceanBase Database. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the cluster_id in oceanbase-ce.
    # cluster_id: 1
    # Zone name for your observer. The default value is zone1. When a depends exists, OBD gets this value from the oceanbase-ce of the depends. The value is the same as the zone name in oceanbase-ce.
    # zone_name: zone1
    # Monitor status for OceanBase Database.  Active is to enable. Inactive is to disable. The default value is active. When you deploy an cluster automatically, OBD decides whether to enable this parameter based on depends.
    ob_monitor_status: active
    # Monitor status for your host. Active is to enable. Inactive is to disable. The default value is active.
    host_monitor_status: inactive
    # Whether to disable the basic authentication for HTTP service. True is to disable. False is to enable. The default value is false.
    disable_http_basic_auth: true
    # Whether to disable the basic authentication for the debug interface. True is to disable. False is to enable. The default value is false.
    disable_pprof_basic_auth: true
ob-configserver:
  servers:
  - 172.17.0.2
  global:
    listen_port: 8080
    server_ip: 0.0.0.0
    home_path: /root/obconfigserver
2 个赞

单独拉起ob试试,或者给memory_limit扩容一下
obd cluster xxx start -c oceanbase-ce

2 个赞

memory_limit 扩容成 8G 或 12G 后启动仍然失败:

2 个赞

log盘和data盘也扩容下,observer日志方便拿出来么

2 个赞

可以的,observer.log 日志如下:

observer.log (3.4 MB)

2 个赞

system_memorty:2G
磁盘路径相应的修改了么
这些可以删掉

# Expiration time for logs. The default value is 7 days.
log_expire_day: 7
# The maximum number for log files. The default value is 10.
log_file_count: 10

看日志显示应该还是参数文件的问题,你可以使用obd demo生成个测试集群。
然后去查看demo的yaml文件进行学习

2 个赞
  • obd demo 也启动失败,这个问题根据 OceanBase 社区 在 Docker 下应该直接就能复现,因为全程都是按照文档执行,没有做过任何自定义。
    • 启动日志:
[root@066d4e23497d /]# obd demo -c oceanbase-ce
Get local repositories ok
Open ssh connection ok
Check for standby tenant ok
Search plugins ok
Cluster status check ok
observer work dir cleaning ok
demo destroyed
oceanbase-ce-4.2.1.8 already installed.
Cluster param config check ok
Open ssh connection ok
Generate observer configuration ok
oceanbase-ce-4.2.1.8 already installed.
+--------------------------------------------------------------------------------------------+
|                                          Packages                                          |
+--------------+---------+------------------------+------------------------------------------+
| Repository   | Version | Release                | Md5                                      |
+--------------+---------+------------------------+------------------------------------------+
| oceanbase-ce | 4.2.1.8 | 108000022024072217.el7 | 499b676f2ede5a16e0c07b2b15991d1160d972e8 |
+--------------+---------+------------------------+------------------------------------------+
Repository integrity check ok
Load param plugin ok
Open ssh connection ok
Parameter check ok
Cluster status check ok
Initializes observer work home ok
Remote oceanbase-ce-4.2.1.8-108000022024072217.el7-499b676f2ede5a16e0c07b2b15991d1160d972e8 repository install ok
Remote oceanbase-ce-4.2.1.8-108000022024072217.el7-499b676f2ede5a16e0c07b2b15991d1160d972e8 repository lib check !!
Try to get lib-repository
oceanbase-ce-libs-4.2.1.8 already installed.
Remote oceanbase-ce-libs-4.2.1.8-108000022024072217.el7-d02f4bfd321370a02550424293beb1be31204038 repository install ok
Remote oceanbase-ce-4.2.1.8-108000022024072217.el7-499b676f2ede5a16e0c07b2b15991d1160d972e8 repository lib check ok
demo deployed
Get local repositories ok
Search plugins ok
Load cluster param plugin ok
Open ssh connection ok
Check before start observer ok
[WARN] OBD-1011: (127.0.0.1) The recommended value of fs.aio-max-nr is 1048576 (Current value: 65536)
[WARN] OBD-1007: (127.0.0.1) The recommended number of core file size is unlimited (Current value: 0)
[WARN] OBD-1007: (127.0.0.1) The recommended number of stack size is unlimited (Current value: 8192)
[WARN] OBD-1017: (127.0.0.1) The value of the "vm.max_map_count" must be within [327600, 1310720] (Current value: 262144, Recommended value: 655360)
[WARN] OBD-1017: (127.0.0.1) The value of the "vm.overcommit_memory" must be 0 (Current value: 1, Recommended value: 0)
[WARN] OBD-1017: (127.0.0.1) The value of the "fs.file-max" must be greater than 6573688 (Current value: 1636382, Recommended value: 6573688)
[WARN] OBD-1012: (127.0.0.1) clog and data use the same disk (/)

Start observer ok
observer program health check x
[WARN] OBD-2002: Failed to start 127.0.0.1 observer
[ERROR] oceanbase-ce start failed
See https://www.oceanbase.com/product/ob-deployer/error-codes .
Trace ID: dd0613a6-6c3c-11ef-ad91-0242ac110002
If you want to view detailed obd logs, please run: obd display-trace dd0613a6-6c3c-11ef-ad91-0242ac110002
  • 参数配置:
oceanbase-ce:
  servers:
  - 127.0.0.1
  global:
    home_path: /root/oceanbase-ce
    appname: demo
    cluster_id: 1725619418
    enable_syslog_recycle: true
    enable_syslog_wf: false
    max_syslog_file_count: 4
    memory_limit: 6144M
    production_mode: false
    __min_full_resource_pool_memory: 1073741824
    system_memory: 1024M
    cpu_count: 16
    datafile_size: 2048M
    datafile_maxsize: 8192M
    datafile_next: 2048M
    log_disk_size: 14336M
2 个赞

obd日志能发出来一份么,默认在 ~./obd/log中

已补充 obd 日志:

obd.log (840.1 KB)

您好, /root/ob/log/observer.log这个日志麻烦提供一下

observer.log 日志已提供:

observer.log (3.4 MB)

有新的日志么 比如重新启动报错后,发一份新的日志

你好问题还存在么,麻烦发一份新的日志

已和研发同学确认,确定是 Docker 的问题,新版 Docker 不再透出 rdtscp 指令集,内核目前依赖该指令集。
该问题仅在 Intel CPU 下出现,Mac M 系列 CPU 没有这一问题。

1 个赞

请问新版Docker问题是不是会导致所有使用inter芯片的电脑都无法按照操作手册做,都会报错。启动步骤windows、MAC都出现了报错。做不下去了