2026-03-09 14:44:42.191 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b027d8d09ff7a557] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=55251, context=Context{parallelIdx=-1, stringMap={ob_log_disk_path=/log1, task_instance_id=56606, task_operation=execute, zone_name=zone3, ob_run_path=/home/admin/obn175/oceanbase, service_version=4.2.1.8, ob_install_path=/home/admin/obn175/oceanbase, ob_sql_port=2881, cluster_id=7, ocpagent_service_name=agent, ob_disk_path_style=OBD, ob_svr_port=2882, sub_task_instance_name=Start observer process, sub_task_instance_id=55251, cluster_name=obn175, target_server_status=RUNNING, former_cluster_status=RUNNING, target_zone_status=RUNNING, former_zone_status=RUNNING, server_id=23, host_id=26, target_cluster_status=RUNNING, ob_run_user=root, latest_execution_start_time=2026-03-09T14:44:42.161+08:00, ob_data_disk_path=/data1}, listMap={server_ids=[23]}}, executor=137.12.7.243 2026-03-09 14:44:42.222 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b027d8d09ff7a557] c.o.o.s.t.b.s.StartObServerProcessTask : begin to start observer process, clusterId=7, obClusterId=1770977335, clusterType=PRIMARY, serverIp=10.16.10.178 2026-03-09 14:44:42.241 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b027d8d09ff7a557] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=26 2026-03-09 14:44:42.253 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b027d8d09ff7a557] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 26 2026-03-09 14:44:42.309 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b027d8d09ff7a557] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.16.10.178:62888/api/v1/process/exists, request body:CheckProcessExistsRequest(name=observer), params:null 2026-03-09 14:44:42.346 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b027d8d09ff7a557] c.o.o.s.t.b.server.ObServerTaskHandler : begin to start observer process, host ip is 10.16.10.178 2026-03-09 14:44:42.369 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b027d8d09ff7a557] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=26 2026-03-09 14:44:42.383 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b027d8d09ff7a557] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 26 2026-03-09 14:44:42.430 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b027d8d09ff7a557] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.16.10.178:62888/api/v1/ob/observer/start, request body:StartObServerProcessRequest(obPath=ObPath(installPath=/home/admin/obn175/oceanbase, dataPath=null, logPath=null, diskPathStyle=null, runPath=/home/admin/obn175/oceanbase), withArguments=false, arguments=null, runUser=root, runGroup=null, readOnly=false), params:null 2026-03-09 14:44:42.506 ERROR 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b027d8d09ff7a557] c.o.o.c.t.e.c.w.subtask.SubtaskExecutor : [AgentClient]:http request is failed, response:Execute shell command failed: failed to execute command: Command{user=root, program=sh, outputType=combined, cmd=cd /home/admin/obn175/oceanbase; ulimit -s 10240 -u 655350 -n 655350 -c unlimited; LD_LIBRARY_PATH=/home/admin/obn175/oceanbase/lib:$LD_LIBRARY_PATH LD_PRELOAD='' /home/admin/obn175/oceanbase/bin/observer, timeout=10s}, exitCode: 164, output: /home/admin/obn175/oceanbase/bin/observer ERROR: current user(uid=0) that starts observer is not the same with the original one(uid=1000), observer starts failed! Fail check_uid_before_start, please use the initial user to start observer! ============= [AFTER_DESTROY] begin to show unstopped thread ============= [AFTER_DESTROY] detect unstopped thread, tid: 13270, name: observer ============= [AFTER_DESTROY] finish to show unstopped thread ============= , error: exit status 164 com.oceanbase.ocp.executor.exception.HttpRequestFailedException: [AgentClient]:http request is failed, response:Execute shell command failed: failed to execute command: Command{user=root, program=sh, outputType=combined, cmd=cd /home/admin/obn175/oceanbase; ulimit -s 10240 -u 655350 -n 655350 -c unlimited; LD_LIBRARY_PATH=/home/admin/obn175/oceanbase/lib:$LD_LIBRARY_PATH LD_PRELOAD='' /home/admin/obn175/oceanbase/bin/observer, timeout=10s}, exitCode: 164, output: /home/admin/obn175/oceanbase/bin/observer ERROR: current user(uid=0) that starts observer is not the same with the original one(uid=1000), observer starts failed! Fail check_uid_before_start, please use the initial user to start observer! ============= [AFTER_DESTROY] begin to show unstopped thread ============= [AFTER_DESTROY] detect unstopped thread, tid: 13270, name: observer ============= [AFTER_DESTROY] finish to show unstopped thread ============= , error: exit status 164 at com.oceanbase.ocp.executor.internal.template.HttpTemplate.chec kSuccess(HttpTemplate.java:476) at com.oceanbase.ocp.executor.internal.template.HttpTemplate.doPost(HttpTemplate.java:286) at com.oceanbase.ocp.executor.internal.template.HttpTemplate.postForNone(HttpTemplate.java:135) at com.oceanbase.ocp.executor.executor.AgentExecutor.startObServerProcess(AgentExecutor.java:349) at com.oceanbase.ocp.obops.internal.cluster.ClusterHostOperationServiceImpl.startObServerProcess(ClusterHostOperationServiceImpl.java:316) at com.oceanbase.ocp.service.task.business.server.ObServerTaskHandler.startObServerProcess(ObServerTaskHandler.java:71) at com.oceanbase.ocp.service.task.business.server.StartObServerProcessTask.run(StartObServerProcessTask.java:61) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.execute(JavaSubtaskRunner.java:64) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.doRun(JavaSubtaskRunner.java:32) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.run(JavaSubtaskRunner.java:26) at com.oceanbase.ocp.core.task.engine.runner.RunnerFactory.doRun(RunnerFactory.java:76) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.doRun(SubtaskExecutor.java:206) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.redirectConsoleOutput(SubtaskExecutor.java:200) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.lambda$submit$2(SubtaskExecutor.java:137) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Set state for subtask: 55251, operation:EXECUTE, state: FAILED 2026-03-09 14:52:33.061 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.c.t.e.runner.JavaSubtaskRunner : Retry subtask, id=55251, context=Context{parallelIdx=-1, stringMap={ob_log_disk_path=/log1, task_instance_id=56606, task_operation=retry, zone_name=zone3, ob_run_path=/home/admin/obn175/oceanbase, service_version=4.2.1.8, ob_install_path=/home/admin/obn175/oceanbase, ob_sql_port=2881, cluster_id=7, ocpagent_service_name=agent, ob_disk_path_style=OBD, ob_svr_port=2882, sub_task_instance_name=Start observer process, sub_task_instance_id=55251, cluster_name=obn175, target_server_status=RUNNING, former_cluster_status=RUNNING, target_zone_status=RUNNING, former_zone_status=RUNNING, server_id=23, host_id=26, target_cluster_status=RUNNING, ob_run_user=root, latest_execution_start_time=2026-03-09T14:52:33.029+08:00, ob_data_disk_path=/data1}, listMap={server_ids=[23]}}, executor=137.12.7.243 2026-03-09 14:52:33.068 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.s.t.b.s.StartObServerProcessTask : begin to kill observer process, serverId=23 2026-03-09 14:52:33.136 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=26 2026-03-09 14:52:33.145 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 26 2026-03-09 14:52:33.187 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.16.10.178:62888/api/v1/process/exists, request body:CheckProcessExistsRequest(name=observer), params:null 2026-03-09 14:52:33.202 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.s.t.b.server.ObServerTaskHandler : no need to stop, observer process is not exist 2026-03-09 14:52:33.209 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.s.t.b.s.StartObServerProcessTask : kill observer process success 2026-03-09 14:52:33.216 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] com.oceanbase.ocp.core.util.TaskUtils : [TaskUtils] wait 3 seconds 2026-03-09 14:52:36.250 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.s.t.b.s.StartObServerProcessTask : begin to start observer process, clusterId=7, obClusterId=1770977335, clusterType=PRIMARY, serverIp=10.16.10.178 2026-03-09 14:52:36.266 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=26 2026-03-09 14:52:36.278 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 26 2026-03-09 14:52:36.325 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.16.10.178:62888/api/v1/process/exists, request body:CheckProcessExistsRequest(name=observer), params:null 2026-03-09 14:52:36.350 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.s.t.b.server.ObServerTaskHandler : begin to start observer process, host ip is 10.16.10.178 2026-03-09 14:52:36.365 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=26 2026-03-09 14:52:36.382 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 26 2026-03-09 14:52:36.435 INFO 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.16.10.178:62888/api/v1/ob/observer/start, request body:StartObServerProcessRequest(obPath=ObPath(installPath=/home/admin/obn175/oceanbase, dataPath=null, logPath=null, diskPathStyle=null, runPath=/home/admin/obn175/oceanbase), withArguments=false, arguments=null, runUser=root, runGroup=null, readOnly=false), params:null 2026-03-09 14:52:36.503 ERROR 8247 --- [manual-subtask-executor14,5012a6974990a3ce,b600a76b275d3d5d] c.o.o.c.t.e.c.w.subtask.SubtaskExecutor : [AgentClient]:http request is failed, response:Execute shell command failed: failed to execute command: Command{user=root, program=sh, outputType=combined, cmd=cd /home/admin/obn175/oceanbase; ulimit -s 10240 -u 655350 -n 655350 -c unlimited; LD_LIBRARY_PATH=/home/admin/obn175/oceanbase/lib:$LD_LIBRARY_PATH LD_PRELOAD='' /home/admin/obn175/oceanbase/bin/observer, timeout=10s}, exitCode: 164, output: /home/admin/obn175/oceanbase/bin/observer ERROR: current user(uid=0) that starts observer is not the same with the original one(uid=1000), observer starts failed! Fail check_uid_before_start, please use the initial user to start observer! ============= [AFTER_DESTROY] begin to show unstopped thread ============= [AFTER_DESTROY] detect unstopped thread, tid: 20962, name: observer ============= [AFTER_DESTROY] finish to show unstopped thread ============= , error: exit status 164 com.oceanbase.ocp.executor.exception.HttpRequestFailedException: [AgentClient]:http request is failed, response:Execute shell command failed: failed to execute command: Command{user=root, program=sh, outputType=combined, cmd=cd /home/admin/obn175/oceanbase; ulimit -s 10240 -u 655350 -n 655350 -c unlimited; LD_LIBRARY_PATH=/home/admin/obn175/oceanbase/lib:$LD_LIBRARY_PATH LD_PRELOAD='' /home/admin/obn175/oceanbase/bin/observer, timeout=10s}, exitCode: 164, output: /home/admin/obn175/oceanbase/bin/observer ERROR: current user(uid=0) that starts observer is not the same with the original one(uid=1000), observer starts failed! Fail check_uid_before_start, please use the initial user to start observer! ============= [AFTER_DESTROY] begin to show unstopped thread ============= [AFTER_DESTROY] detect unstopped thread, tid: 20962, name: observer ============= [AFTER_DESTROY] finish to show unstopped thread ============= , error: exit status 164 at com.oceanbase.ocp.executor.internal.template.HttpTemplate.chec kSuccess(HttpTemplate.java:476) at com.oceanbase.ocp.executor.internal.template.HttpTemplate.doPost(HttpTemplate.java:286) at com.oceanbase.ocp.executor.internal.template.HttpTemplate.postForNone(HttpTemplate.java:135) at com.oceanbase.ocp.executor.executor.AgentExecutor.startObServerProcess(AgentExecutor.java:349) at com.oceanbase.ocp.obops.internal.cluster.ClusterHostOperationServiceImpl.startObServerProcess(ClusterHostOperationServiceImpl.java:316) at com.oceanbase.ocp.service.task.business.server.ObServerTaskHandler.startObServerProcess(ObServerTaskHandler.java:71) at com.oceanbase.ocp.service.task.business.server.StartObServerProcessTask.run(StartObServerProcessTask.java:61) at com.oceanbase.ocp.core.task.runtime.Subtask.retry(Subtask.java:49) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.retry(JavaSubtaskRunner.java:76) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.doRun(JavaSubtaskRunner.java:35) at com.oceanbase.ocp.core.task.engine.runner.JavaSubtaskRunner.run(JavaSubtaskRunner.java:26) at com.oceanbase.ocp.core.task.engine.runner.RunnerFactory.doRun(RunnerFactory.java:76) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.doRun(SubtaskExecutor.java:206) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.redirectConsoleOutput(SubtaskExecutor.java:200) at com.oceanbase.ocp.core.task.engine.coordinator.worker.subtask.SubtaskExecutor.lambda$submit$2(SubtaskExecutor.java:137) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Set state for subtask: 55251, operation:RETRY, state: FAILED