ocp-4.2.1-ce容器启动失败

达瓦里希1001 · 2024 年2 月 28 日 17:47

【使用环境】生产环境
【 OB or 其他组件】OCP
【使用版本】oceanbase/ocp-ce 4.2.1 703dcc3d8669
“Created”: “2023-12-08T15:22:28.877578395+08:00”
【问题描述】使用docker run 启动容器后，web端一直无响应，登录容器内查看，发现bootstrap失败。
【复现路径】问题出现前后相关操作
【附件及日志】

ocp-server.0.err日志如下:

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
Caused by: java.lang.IllegalArgumentException: Cannot instantiate interface org.springframework.boot.SpringApplicationRunListener : com.oceanbase.ocp.bootstrap.spring.BootstrapRunListener
        at org.springframework.boot.SpringApplication.createSpringFactoriesInstances(SpringApplication.java:449)
        at org.springframework.boot.SpringApplication.getSpringFactoriesInstances(SpringApplication.java:431)
        at org.springframework.boot.SpringApplication.getRunListeners(SpringApplication.java:419)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:297)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292)
        at com.oceanbase.ocp.OcpCeServerApplication.main(OcpCeServerApplication.java:20)
        ... 8 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.oceanbase.ocp.bootstrap.spring.BootstrapRunListener]: Constructor threw exception; nested exception is java.lang.NumberFormatException: For input string: "8080 ocp.site.url:http://172.21.182.231:8080 obsdk.ob.connection.mode:direct"
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:224)
        at org.springframework.boot.SpringApplication.createSpringFactoriesInstances(SpringApplication.java:445)
        ... 14 more
Caused by: java.lang.NumberFormatException: For input string: "8080 ocp.site.url:http://172.21.182.231:8080 obsdk.ob.connection.mode:direct"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Integer.parseInt(Integer.java:580)
        at java.lang.Integer.parseInt(Integer.java:615)
        at com.oceanbase.ocp.bootstrap.Bootstrap.getOcpPort(Bootstrap.java:216)
        at com.oceanbase.ocp.bootstrap.hooks.BootstrapLock.clientId(BootstrapLock.java:62)
        at com.oceanbase.ocp.bootstrap.hooks.BootstrapLock.getInstance(BootstrapLock.java:56)
        at com.oceanbase.ocp.bootstrap.hooks.OCPInitializer.lockBootstrap(OCPInitializer.java:188)
        at com.oceanbase.ocp.bootstrap.hooks.OCPInitializer.initialize(OCPInitializer.java:65)
        at com.oceanbase.ocp.bootstrap.hooks.OCPInitializer.initialize(OCPInitializer.java:110)
at com.oceanbase.ocp.bootstrap.spring.BootstrapRunListener.<init>(BootstrapRunListener.java:59)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
        ... 15 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
Caused by: java.lang.IllegalArgumentException: Cannot instantiate interface org.springframework.boot.SpringApplicationRunListener : com.oceanbase.ocp.bootstrap.spring.BootstrapRunListener
        at org.springframework.boot.SpringApplication.createSpringFactoriesInstances(SpringApplication.java:449)
        at org.springframework.boot.SpringApplication.getSpringFactoriesInstances(SpringApplication.java:431)
        at org.springframework.boot.SpringApplication.getRunListeners(SpringApplication.java:419)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:297)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292)
        at com.oceanbase.ocp.OcpCeServerApplication.main(OcpCeServerApplication.java:20)
        ... 8 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.oceanbase.ocp.bootstrap.spring.BootstrapRunListener]: Constructor threw exception; nested exception is java.lang.NumberFormatException: For input string: "8080 ocp.site.url:http://172.21.182.231:8080 obsdk.ob.connection.mode:direct"
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:224)
        at org.springframework.boot.SpringApplication.createSpringFactoriesInstances(SpringApplication.java:445)
        ... 14 more
Caused by: java.lang.NumberFormatException: For input string: "8080 ocp.site.url:http://172.21.182.231:8080 obsdk.ob.connection.mode:direct"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Integer.parseInt(Integer.java:580)
        at java.lang.Integer.parseInt(Integer.java:615)
        at com.oceanbase.ocp.bootstrap.Bootstrap.getOcpPort(Bootstrap.java:216)
        at com.oceanbase.ocp.bootstrap.hooks.BootstrapLock.clientId(BootstrapLock.java:62)
        at com.oceanbase.ocp.bootstrap.hooks.BootstrapLock.getInstance(BootstrapLock.java:56)
        at com.oceanbase.ocp.bootstrap.hooks.OCPInitializer.lockBootstrap(OCPInitializer.java:188)
        at com.oceanbase.ocp.bootstrap.hooks.OCPInitializer.initialize(OCPInitializer.java:65)
        at com.oceanbase.ocp.bootstrap.hooks.OCPInitializer.initialize(OCPInitializer.java:110)
        at com.oceanbase.ocp.bootstrap.spring.BootstrapRunListener.<init>(BootstrapRunListener.java:59)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
        ... 15 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
Caused by: java.lang.IllegalArgumentException: Cannot instantiate interface org.springframework.boot.SpringApplicationRunListener : com.oceanbase.ocp.bootstrap.spring.BootstrapRunListener
        at org.springframework.boot.SpringApplication.createSpringFactoriesInstances(SpringApplication.java:449)
        at org.springframework.boot.SpringApplication.getSpringFactoriesInstances(SpringApplication.java:431)
        at org.springframework.boot.SpringApplication.getRunListeners(SpringApplication.java:419)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:297)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292)
        at com.oceanbase.ocp.OcpCeServerApplication.main(OcpCeServerApplication.java:20)
        ... 8 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.oceanbase.ocp.bootstrap.spring.BootstrapRunListener]: Constructor threw exception; nested exception is java.lang.NumberFormatException: For input string: "8080 ocp.site.url:http://172.21.182.231:8080 obsdk.ob.connection.mode:direct"
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:224)
        at org.springframework.boot.SpringApplication.createSpringFactoriesInstances(SpringApplication.java:445)
        ... 14 more
Caused by: java.lang.NumberFormatException: For input string: "8080 ocp.site.url:http://172.21.182.231:8080 obsdk.ob.connection.mode:direct"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Integer.parseInt(Integer.java:580)
        at java.lang.Integer.parseInt(Integer.java:615)
        at com.oceanbase.ocp.bootstrap.Bootstrap.getOcpPort(Bootstrap.java:216)
        at com.oceanbase.ocp.bootstrap.hooks.BootstrapLock.clientId(BootstrapLock.java:62)
        at com.oceanbase.ocp.bootstrap.hooks.BootstrapLock.getInstance(BootstrapLock.java:56)
        at com.oceanbase.ocp.bootstrap.hooks.OCPInitializer.lockBootstrap(OCPInitializer.java:188)
        at com.oceanbase.ocp.bootstrap.hooks.OCPInitializer.initialize(OCPInitializer.java:65)
        at com.oceanbase.ocp.bootstrap.hooks.OCPInitializer.initialize(OCPInitializer.java:110)
        at com.oceanbase.ocp.bootstrap.spring.BootstrapRunListener.<init>(BootstrapRunListener.java:59)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
        ... 15 more
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)

ocp-server.0.out 中无ERROR日志，截取最后部分的日志

.....
17:44:33.323 [main] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from core/spring_jdbc_session.yaml
17:44:33.325 [main] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from monitor/monitor_metric_ob_collect_config.yaml
17:44:33.351 [ForkJoinPool.commonPool-worker-1] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from mc/mc_core_tables.yaml
17:44:33.389 [ForkJoinPool.commonPool-worker-1] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from obproxy/obproxy_data.yaml
17:44:33.393 [ForkJoinPool.commonPool-worker-1] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from common/task_data.yaml
17:44:33.399 [main] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from monitor/metric_migrations.yaml
17:44:33.401 [main] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from monitor/monitor_collect_item_detail_ob.yaml
17:44:33.404 [ForkJoinPool.commonPool-worker-2] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from alert/alarm_rule_collect_group.yaml
17:44:33.411 [ForkJoinPool.commonPool-worker-2] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from alert/ocp2_alarm_rule.yaml
17:44:33.448 [main] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from monitor/metric_daily_migration.yaml
17:44:33.451 [main] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from common/common_tables.yaml
17:44:33.453 [ForkJoinPool.commonPool-worker-1] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from alert/ocp_alarm_i18n.yaml
17:44:33.459 [ForkJoinPool.commonPool-worker-1] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from obcloud/obcloud_tables.yaml
17:44:33.489 [ForkJoinPool.commonPool-worker-1] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from inspection/inspection_data.yaml
17:44:33.491 [ForkJoinPool.commonPool-worker-3] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from ob/ob_tables.yaml
17:44:33.538 [main] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from ob/ob_data.yaml
17:44:33.554 [main] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from ob/ob_cluster_parameter_template_info.yaml
17:44:33.590 [main] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from ob/ocp_ob_error_code.yaml
17:44:33.602 [ForkJoinPool.commonPool-worker-3] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from cmdb/cmdb_tables.yaml
17:44:33.606 [ForkJoinPool.commonPool-worker-3] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from cmdb/cmdb_config.yaml
17:44:33.614 [ForkJoinPool.commonPool-worker-3] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from config_properties_ce.yaml
17:44:33.617 [ForkJoinPool.commonPool-worker-3] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from inspection/inspection_tables.yaml
17:44:33.648 [ForkJoinPool.commonPool-worker-3] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from user/user_tables.yaml
17:44:33.717 [ForkJoinPool.commonPool-worker-3] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from diag/diag_tables.yaml
17:44:33.901 [main] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from user/profile_credential.yaml
17:44:33.963 [ForkJoinPool.commonPool-worker-2] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from mc/mc_ocp_cluster_config.yaml
17:44:34.074 [ForkJoinPool.commonPool-worker-1] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from alert/ocp2_template_variable_meta.yaml
17:44:34.091 [ForkJoinPool.commonPool-worker-1] INFO com.oceanbase.ocp.bootstrap.core.config.DataConfigLoader - load config from mc/mc_tables.yaml
17:44:34.174 [main] INFO com.oceanbase.ocp.bootstrap.hooks.OCPInitializer - run initialize. action: UNKNOWN
17:44:34.700 [main] INFO com.alibaba.druid.pool.DruidDataSource - {dataSource-1} inited
17:44:34.767 [main] INFO com.oceanbase.ocp.bootstrap.hooks.OCPInitializer - automatically choosing bootstrap action. old null -> new 4.2.1-20231208144448
17:44:34.767 [main] INFO com.oceanbase.ocp.bootstrap.hooks.OCPInitializer - meta db is empty, choose INSTALL
17:44:34.787 [main] INFO com.oceanbase.ocp.common.util.HostUtils - Get local ip by meta server socket, jdb url:jdbc:oceanbase://172.21.182.231:2881/ocp_meta
17:44:34.790 [main] INFO com.oceanbase.ocp.common.util.HostUtils - Get local ip by meta server socket success, local ip:172.21.182.231

/home/admin/logs/ocp 目录下，bootstrap.log为空，看起来没有发起引导。不过这里是部署单节点的OCP我理解没有bootstrap是不是也是合理的？

[root@cwb-182-231 ocp]# ll
total 4
-rw-r--r-- 1 admin admin    0 2月  28 17:31 bootstrap.log
-rw-r--r-- 1 admin admin 3564 2月  28 17:45 gc.log.0.current
[root@cwb-182-231 ocp]# pwd
/home/admin/logs/ocp

达瓦里希1001 · 2024 年3 月 1 日 12:13

自己给自己回复吧，原因排查到了，是按照手册设置环境变量的时候，要特别注意

export OCP_CONFIG_PROPERTIES=`cat << EOF
server.port:8080
ocp.site.url:http://172.21.182.231:8080
obsdk.ob.connection.mode:direct
EOF
`

启动的时候，最好是用文档推荐的方式：

docker run -d --name ocp-421 \
--network host \ 
--cpu-period 100000 --cpu-quota ${OCP_CPU_COUNT}00000 --memory=${OCP_MEMORY_GB}G \
 -e OCP_METADB_HOST="${OCP_METADB_HOST}" \
 -e OCP_METADB_PORT="${OCP_METADB_PORT}" \
 -e OCP_METADB_DBNAME="${OCP_METADB_DBNAME}" \
 -e OCP_METADB_USER="${OCP_METADB_USER}" \
 -e OCP_METADB_PASSWORD="${OCP_METADB_PASSWORD}" \
 -e OCP_MONITORDB_DBNAME="${OCP_MONITORDB_DBNAME}" \
 -e OCP_MONITORDB_USER="${OCP_MONITORDB_USER}" \
 -e OCP_MONITORDB_PASSWORD="${OCP_MONITORDB_PASSWORD}" \
 -e OCP_INITIAL_ADMIN_PASSWORD="${OCP_INITIAL_ADMIN_PASSWORD}" \
 -e OCP_CONFIG_PROPERTIES="${OCP_CONFIG_PROPERTIES}" \
oceanbase/ocp-ce:4.2.1

如果按照https://ask.oceanbase.com/t/topic/35606283/11

这个贴子里面的方式启动容器，

docker run -d --name ocp-421 --network host --cpu-period 100000 --cpu-quota 400000 --memory=8G -e OCP_METADB_HOST=“172.xx.xx.86” -e OCP_METADB_PORT=“5432” -e OCP_METADB_DBNAME=“meta” -e OCP_METADB_USER=“oot@meta_tenant#obcluster” -e OCP_METADB_PASSWORD=“Meta_xx” -e OCP_MONITORDB_DBNAME=“monitor” -e OCP_MONITORDB_USER=“root@monitor_tenant#obcluster” -e OCP_MONITORDB_PASSWORD=“Monitor_xx” -e OCP_CONFIG_PROPERTIES=“cat << EOF
server.port:5433 ocp.site.url:http://172.xx.xx.86:5433 obsdk.ob.connection.mode:direct EOF” oceanbase/ocp-ce:4.2.1

容易导致OCP_CONFIG_PROPERTIES被截断，出现本贴的报错现象