2025-03-02 11:41:52.884 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,724e5a7ddb12e58a] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-03-02 11:41:52.888 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,724e5a7ddb12e58a] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=217, context=Context{parallelIdx=0, stringMap={ob_log_disk_path=/data/log1, primary_root_sys_password=xxx task_instance_id=24, task_operation=execute, ob_run_path=/home/admin/oceanbase, cluster_type=PRIMARY, ob_install_path=/home/admin/oceanbase, obproxy_proxyro_username=proxyro, ob_sql_port=5881, cluster_id=3, operation_scope=Cluster, ob_svr_port=5882, sub_task_instance_name=Pre check for install ob, sub_task_instance_id=217, cluster_version=4.3.5.0, cluster_name=bigdemo, target_server_status=RUNNING, mix_rpm_names={"zone1":"oceanbase-ce-4.3.5.0-100000202024123117.el7.x86_64.rpm"}, startup_option_string=large_query_threshold=600s,max_syslog_file_count=300,enable_syslog_recycle=1,enable_cgroup=true,enable_record_trace_log=false, subtask_splitter=host_ids, ob_version=4.3.5.0, service_name=bigdemo:3, target_zone_status=RUNNING, ob_cluster_id=3, zone1.idc=cq, service_type=OB_CLUSTER, obproxy_proxyro_password=xxx root_sys_password=xxx dep_rpm_names={"zone1":["oceanbase-ce-libs-4.3.5.0-100000202024123117.el7.x86_64.rpm","oceanbase-ce-utils-4.3.5.0-100000202024123117.el7.x86_64.rpm"]}, ob_run_user=admin, target_cluster_status=RUNNING, latest_execution_start_time=2025-03-02T11:41:52.865+08:00, ob_data_disk_path=/data/1, mix_obs_rpm={}}, listMap={root_servers=[3], server_ids=[7], host_ids=[3], zone1.host_ids=[3], zone_names=[zone1]}}, executor=192.168.2.161 2025-03-02 11:41:52.901 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,724e5a7ddb12e58a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-03-02 11:41:52.908 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,724e5a7ddb12e58a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-03-02 11:41:52.956 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,724e5a7ddb12e58a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://192.168.2.164:62888/api/v1/ob/install/preCheck, request body:ObClusterInstallRequest(obClusterName=bigdemo, obVersion=4.3.5.0, obPath=ObPath(installPath=/home/admin/oceanbase, dataPath=/data/1, logPath=/data/log1, diskPathStyle=null, runPath=/home/admin/oceanbase), runUser=null), params:null 2025-03-02 11:41:52.964 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,724e5a7ddb12e58a] .o.o.i.c.ClusterHostOperationServiceImpl : ob install pre check for observer, expect process should not exist, passed=false 2025-03-02 11:41:52.970 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,724e5a7ddb12e58a] .o.o.i.c.ClusterHostOperationServiceImpl : ob install pre check for /data/1/bigdemo/sstable/block_file, expect file should not exist, passed=true 2025-03-02 11:41:52.973 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,724e5a7ddb12e58a] .o.o.i.c.ClusterHostOperationServiceImpl : ob install pre check for /data/log1/bigdemo/clog, expect file should not exist, passed=true 2025-03-02 11:41:52.975 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,724e5a7ddb12e58a] .o.o.i.c.ClusterHostOperationServiceImpl : ob install pre check 1 items failed 2025-03-02 11:41:52.978 ERROR 10189 --- [manual-subtask-executor15,c122d01a99068063,724e5a7ddb12e58a] c.o.o.c.t.e.c.w.subtask.SubtaskExecutor : An unknown error has occurred. Cause: ob install pre check failed. Error message: ob install pre check failed, maybe from another observer, please check. Contact the administrator. com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=ob install pre check failed,ob install pre check failed, maybe from another observer, please check at com.oceanbase.ocp.core.i18n.ErrorCodes$Kind$10.exception(ErrorCodes.java:1774) at com.oceanbase.ocp.core.i18n.ErrorCodes.exception(ErrorCodes.java:1629) at com.oceanbase.ocp.core.util.ExceptionUtils.require(ExceptionUtils.java:154) at com.oceanbase.ocp.obops.internal.cluster.ClusterHostOperationServiceImpl.obInstallPreCheck(ClusterHostOperationServiceImpl.java:113) at com.oceanbase.ocp.service.task.business.cluster.InstallPreCheckTask.run(InstallPreCheckTask.java:40) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.execute(JavaSubtaskRunner.java:64) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.doRun(JavaSubtaskRunner.java:32) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.run(JavaSubtaskRunner.java:26) at com.oceanbase.ocp.core.task.engine.runner.RunnerFactory.doRun(RunnerFactory.java:76) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.doRun(SubtaskExecutor.java:206) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.redirectConsoleOutput(SubtaskExecutor.java:200) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.lambda$submit$2(SubtaskExecutor.java:137) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Th read.run(Thread.java:750) Set state for subtask: 217, operation:EXECUTE, state: FAILED 2025-03-02 11:42:10.941 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-03-02 11:42:10.945 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] c.o.o.c.t.e.runner.JavaSubtaskRunner : Retry subtask, id=217, context=Context{parallelIdx=0, stringMap={ob_log_disk_path=/data/log1, primary_root_sys_password=xxx task_instance_id=24, task_operation=retry, ob_run_path=/home/admin/oceanbase, cluster_type=PRIMARY, ob_install_path=/home/admin/oceanbase, obproxy_proxyro_username=proxyro, ob_sql_port=5881, cluster_id=3, operation_scope=Cluster, ob_svr_port=5882, sub_task_instance_name=Pre check for install ob, sub_task_instance_id=217, cluster_version=4.3.5.0, cluster_name=bigdemo, target_server_status=RUNNING, mix_rpm_names={"zone1":"oceanbase-ce-4.3.5.0-100000202024123117.el7.x86_64.rpm"}, startup_option_string=large_query_threshold=600s,max_syslog_file_count=300,enable_syslog_recycle=1,enable_cgroup=true,enable_record_trace_log=false, subtask_splitter=host_ids, ob_version=4.3.5.0, service_name=bigdemo:3, target_zone_status=RUNNING, ob_cluster_id=3, zone1.idc=cq, service_type=OB_CLUSTER, obproxy_proxyro_password=xxx root_sys_password=xxx dep_rpm_names={"zone1":["oceanbase-ce-libs-4.3.5.0-100000202024123117.el7.x86_64.rpm","oceanbase-ce-utils-4.3.5.0-100000202024123117.el7.x86_64.rpm"]}, ob_run_user=admin, target_cluster_status=RUNNING, latest_execution_start_time=2025-03-02T11:42:10.920+08:00, ob_data_disk_path=/data/1, mix_obs_rpm={}}, listMap={root_servers=[3], server_ids=[7], host_ids=[3], zone1.host_ids=[3], zone_names=[zone1]}}, executor=192.168.2.161 2025-03-02 11:42:10.948 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] c.o.o.s.t.b.cluster.InstallPreCheckTask : need to do nothing when rollback 2025-03-02 11:42:10.959 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-03-02 11:42:10.964 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-03-02 11:42:11.002 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://192.168.2.164:62888/api/v1/ob/install/preCheck, request body:ObClusterInstallRequest(obClusterName=bigdemo, obVersion=4.3.5.0, obPath=ObPath(installPath=/home/admin/oceanbase, dataPath=/data/1, logPath=/data/log1, diskPathStyle=null, runPath=/home/admin/oceanbase), runUser=null), params:null 2025-03-02 11:42:11.011 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] .o.o.i.c.ClusterHostOperationServiceImpl : ob install pre check for observer, expect process should not exist, passed=false 2025-03-02 11:42:11.013 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] .o.o.i.c.ClusterHostOperationServiceImpl : ob install pre check for /data/1/bigdemo/sstable/block_file, expect file should not exist, passed=true 2025-03-02 11:42:11.016 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] .o.o.i.c.ClusterHostOperationServiceImpl : ob install pre check for /data/log1/bigdemo/clog, expect file should not exist, passed=true 2025-03-02 11:42:11.019 INFO 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] .o.o.i.c.ClusterHostOperationServiceImpl : ob install pre check 1 items failed 2025-03-02 11:42:11.021 ERROR 10189 --- [manual-subtask-executor15,c122d01a99068063,b301a03718fb2b33] c.o.o.c.t.e.c.w.subtask.SubtaskExecutor : An unknown error has occurred. Cause: ob install pre check failed. Error message: ob install pre check failed, maybe from another observer, please check. Contact the administrator. com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=ob install pre check failed,ob install pre check failed, maybe from another observer, please check at com.oceanbase.ocp.core.i18n.ErrorCodes$Kind$10.exception(ErrorCodes.java:1774) at com.oceanbase.ocp.core.i18n.ErrorCodes.exception(ErrorCodes.java:1629) at com.oceanbase.ocp.core.util.ExceptionUtils.require(ExceptionUtils.java:154) at com.oceanbase.ocp.obops.internal.cluster.ClusterHostOperationServiceImpl.obInstallPreCheck(ClusterHostOperationServiceImpl.java:113) at com.oceanbase.ocp.service.task.business.cluster.InstallPreCheckTask.run(InstallPreCheckTask.java:40) at com.oceanbase.ocp.core.task.runtime.Subtask.retry(Subtask.java:49) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.retry(JavaSubtaskRunner.java:76) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.doRun(JavaSubtaskRunner.java:35) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.run(JavaSubtaskRunner.java:26) at com.oceanbase.ocp.core.task.engine.runner.RunnerFactory.doRun(RunnerFactory.java:76) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.doRun(SubtaskExecutor.java:206) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.redirectConsoleOutput(SubtaskExecutor.java:200) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.lambda$submit$2(SubtaskExecutor.java:137) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.Thread PoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Set state for subtask: 217, operation:RETRY, state: FAILED