2025-11-29 08:41:30.505 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=11, ip=10.202.22.47, timestamp=2025-11-29T08:41:30.504755700+08:00 2025-11-29 08:41:30.505 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/time, params:null 2025-11-29 08:41:30.516 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-11-29T08:41:30.505+08:00 2025-11-29 08:41:30.517 INFO 12717 --- [host-check-agent-status9,0b8def41858b64cc,09aa33b360112d4f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.61 2025-11-29 08:41:30.517 INFO 12717 --- [host-check-agent-status5,0b8def41858b64cc,f82d9504803e7213] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.53 2025-11-29 08:41:30.517 INFO 12717 --- [host-check-agent-status6,0b8def41858b64cc,1778b4c995e12753] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.54 2025-11-29 08:41:30.517 INFO 12717 --- [host-check-agent-status9,0b8def41858b64cc,09aa33b360112d4f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.61 2025-11-29 08:41:30.517 INFO 12717 --- [host-check-agent-status6,0b8def41858b64cc,1778b4c995e12753] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.54 2025-11-29 08:41:30.517 INFO 12717 --- [host-check-agent-status6,0b8def41858b64cc,1778b4c995e12753] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.54 2025-11-29 08:41:30.517 INFO 12717 --- [host-check-agent-status9,0b8def41858b64cc,09aa33b360112d4f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.61 2025-11-29 08:41:30.517 INFO 12717 --- [host-check-agent-status5,0b8def41858b64cc,f82d9504803e7213] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.53 2025-11-29 08:41:30.517 INFO 12717 --- [host-check-agent-status5,0b8def41858b64cc,f82d9504803e7213] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.53 2025-11-29 08:41:30.518 INFO 12717 --- [host-check-agent-status1,0b8def41858b64cc,5a44dffd9f2055c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.58 2025-11-29 08:41:30.518 INFO 12717 --- [host-check-agent-status1,0b8def41858b64cc,5a44dffd9f2055c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.58 2025-11-29 08:41:30.518 INFO 12717 --- [host-check-agent-status1,0b8def41858b64cc,5a44dffd9f2055c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.58 2025-11-29 08:41:30.518 INFO 12717 --- [host-check-agent-status4,0b8def41858b64cc,a85899dd185b6268] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.52 2025-11-29 08:41:30.518 INFO 12717 --- [host-check-agent-status4,0b8def41858b64cc,a85899dd185b6268] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.52 2025-11-29 08:41:30.518 INFO 12717 --- [host-check-agent-status4,0b8def41858b64cc,a85899dd185b6268] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.52 2025-11-29 08:41:30.518 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2025-11-29T08:41:30.502+08:00 and status AVAILABLE 2025-11-29 08:41:30.518 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2025-11-29T08:41:30.502+08:00 and status AVAILABLE 2025-11-29 08:41:30.518 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-11-29T08:41:30.502+08:00 and status AVAILABLE 2025-11-29 08:41:30.519 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-11-29T08:41:30.504+08:00 2025-11-29 08:41:30.520 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=11, lastAvailableTime=2025-11-29T08:41:30.505+08:00 2025-11-29 08:41:30.520 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-11-29T08:41:30.505+08:00 and status AVAILABLE 2025-11-29 08:41:30.521 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/time, params:null 2025-11-29 08:41:30.533 INFO 12717 --- [host-check-agent-status10,0b8def41858b64cc,140c3afaa3f94f50] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.47 2025-11-29 08:41:30.533 INFO 12717 --- [host-check-agent-status10,0b8def41858b64cc,140c3afaa3f94f50] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.47 2025-11-29 08:41:30.533 INFO 12717 --- [host-check-agent-status10,0b8def41858b64cc,140c3afaa3f94f50] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.47 2025-11-29 08:41:30.533 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.s.compute.HostAgentDaoManager : Updated agent 11 last available time 2025-11-29T08:41:30.505+08:00 and status AVAILABLE 2025-11-29 08:41:30.533 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.s.compute.HostAgentDaoManager : Updated agent 10 last available time 2025-11-29T08:41:30.504+08:00 and status AVAILABLE 2025-11-29 08:41:30.534 INFO 12717 --- [host-check-agent-status3,0b8def41858b64cc,1e24df1341024f18] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/agent/status, params:null 2025-11-29 08:41:30.534 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-11-29T08:41:30.504+08:00 and status AVAILABLE 2025-11-29 08:41:30.535 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.202.22.52, timestamp=2025-11-29T08:41:30.519315822+08:00 2025-11-29 08:41:30.535 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=12, ip=10.202.22.48, timestamp=2025-11-29T08:41:30.532553172+08:00 2025-11-29 08:41:30.535 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=13, ip=10.202.22.49, timestamp=2025-11-29T08:41:30.533543643+08:00 2025-11-29 08:41:30.535 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=12, lastAvailableTime=2025-11-29T08:41:30.535+08:00 2025-11-29 08:41:30.535 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-11-29T08:41:30.535+08:00 2025-11-29 08:41:30.536 INFO 12717 --- [host-check-agent-status2,0b8def41858b64cc,4e8fad091c726a39] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/agent/status, params:null 2025-11-29 08:41:30.536 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=13, lastAvailableTime=2025-11-29T08:41:30.535+08:00 2025-11-29 08:41:30.536 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/time, params:null 2025-11-29 08:41:30.537 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-11-29 08:41:30.537 INFO 12717 --- [host-check-agent-status8,0b8def41858b64cc,10ffb5d566661676] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/agent/status, params:null 2025-11-29 08:41:30.537 INFO 12717 --- [host-check-agent-status3,0b8def41858b64cc,1e24df1341024f18] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.51 2025-11-29 08:41:30.537 INFO 12717 --- [host-check-agent-status3,0b8def41858b64cc,1e24df1341024f18] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.51 2025-11-29 08:41:30.537 INFO 12717 --- [host-check-agent-status3,0b8def41858b64cc,1e24df1341024f18] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.51 2025-11-29 08:41:30.538 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.202.22.53, timestamp=2025-11-29T08:41:30.537086174+08:00 2025-11-29 08:41:30.539 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.s.compute.HostAgentDaoManager : Updated agent 13 last available time 2025-11-29T08:41:30.535+08:00 and status AVAILABLE 2025-11-29 08:41:30.539 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.s.compute.HostAgentDaoManager : Updated agent 12 last available time 2025-11-29T08:41:30.535+08:00 and status AVAILABLE 2025-11-29 08:41:30.539 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2025-11-29 08:41:30.539 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=10 2025-11-29 08:41:30.539 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-11-29 08:41:30.539 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-11-29T08:41:30.535+08:00 and status AVAILABLE 2025-11-29 08:41:30.540 INFO 12717 --- [host-check-agent-status2,0b8def41858b64cc,4e8fad091c726a39] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.50 2025-11-29 08:41:30.540 INFO 12717 --- [host-check-agent-status2,0b8def41858b64cc,4e8fad091c726a39] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.50 2025-11-29 08:41:30.540 INFO 12717 --- [host-check-agent-status2,0b8def41858b64cc,4e8fad091c726a39] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.50 2025-11-29 08:41:30.540 INFO 12717 --- [host-check-agent-status12,0b8def41858b64cc,e6a74b1520ed876c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/agent/status, params:null 2025-11-29 08:41:30.540 INFO 12717 --- [host-check-agent-status8,0b8def41858b64cc,10ffb5d566661676] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.60 2025-11-29 08:41:30.540 INFO 12717 --- [host-check-agent-status8,0b8def41858b64cc,10ffb5d566661676] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.60 2025-11-29 08:41:30.540 INFO 12717 --- [host-check-agent-status8,0b8def41858b64cc,10ffb5d566661676] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.60 2025-11-29 08:41:30.540 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-11-29T08:41:30.538+08:00 2025-11-29 08:41:30.541 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-11-29 08:41:30.541 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=9 2025-11-29 08:41:30.542 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=11 2025-11-29 08:41:30.544 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-11-29T08:41:30.538+08:00 and status AVAILABLE 2025-11-29 08:41:30.544 INFO 12717 --- [host-check-agent-status12,0b8def41858b64cc,e6a74b1520ed876c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.49 2025-11-29 08:41:30.544 INFO 12717 --- [host-check-agent-status12,0b8def41858b64cc,e6a74b1520ed876c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.49 2025-11-29 08:41:30.544 INFO 12717 --- [host-check-agent-status12,0b8def41858b64cc,e6a74b1520ed876c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.49 2025-11-29 08:41:30.547 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=13 2025-11-29 08:41:30.547 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=12 2025-11-29 08:41:30.548 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-11-29 08:41:30.554 INFO 12717 --- [host-check-agent-status11,0b8def41858b64cc,7cd22d7f60f439e1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.48 2025-11-29 08:41:30.555 INFO 12717 --- [host-check-agent-status11,0b8def41858b64cc,7cd22d7f60f439e1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.48 2025-11-29 08:41:30.555 INFO 12717 --- [host-check-agent-status11,0b8def41858b64cc,7cd22d7f60f439e1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.48 2025-11-29 08:41:30.569 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-11-29 08:41:30.573 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:41:30.573 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:41:30.573 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:41:30.575 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:41:30.575 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:41:30.575 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:41:30.575 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:41:30.576 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:41:30.577 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:41:30.577 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:41:30.578 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:41:30.578 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:41:30.579 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:41:30.579 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:41:30.579 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:41:30.579 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:41:30.580 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:41:30.581 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:41:30.581 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:41:30.582 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:41:30.582 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=8, ip=10.202.22.59, timestamp=2025-11-29T08:41:30.581024957+08:00 2025-11-29 08:41:30.583 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=8, lastAvailableTime=2025-11-29T08:41:30.582+08:00 2025-11-29 08:41:30.586 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.s.compute.HostAgentDaoManager : Updated agent 8 last available time 2025-11-29T08:41:30.582+08:00 and status AVAILABLE 2025-11-29 08:41:30.587 INFO 12717 --- [Check status for all clusters2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.3.5.1 2025-11-29 08:41:30.587 INFO 12717 --- [host-check-agent-status7,0b8def41858b64cc,46226a7c61dfdfbc] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.59 2025-11-29 08:41:30.587 INFO 12717 --- [host-check-agent-status7,0b8def41858b64cc,46226a7c61dfdfbc] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.59 2025-11-29 08:41:30.587 INFO 12717 --- [host-check-agent-status7,0b8def41858b64cc,46226a7c61dfdfbc] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.59 2025-11-29 08:41:30.589 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:41:30.591 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=8 2025-11-29 08:41:30.593 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:41:30.598 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.598 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.598 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.601 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:41:30.601 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.602 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.602 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:41:30.602 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.604 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.605 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.607 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.620 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.629 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.631 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:41:30.650 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-11-29 08:41:30.652 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-11-29 08:41:30.656 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-11-29 08:41:30.662 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=16 2025-11-29 08:41:30.663 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=34 2025-11-29 08:41:30.670 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-11-29 08:41:30.671 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-11-29 08:41:30.672 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=16 2025-11-29 08:41:30.672 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-11-29 08:41:30.673 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=34 2025-11-29 08:41:30.673 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-11-29 08:41:30.674 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=31 2025-11-29 08:41:30.682 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-11-29 08:41:30.685 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=22 2025-11-29 08:41:30.685 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=31 2025-11-29 08:41:30.693 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-11-29 08:41:30.702 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=22 2025-11-29 08:41:30.709 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-11-29 08:41:30.775 INFO 12717 --- [prometheus-second-schedule-1,718847a2758df6d5,be05088c42b949b6] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=42, collectTaskCount=42 2025-11-29 08:41:30.796 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=37 2025-11-29 08:41:30.805 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=37 2025-11-29 08:41:30.871 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.882 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.882 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.883 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.883 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.884 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.884 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.885 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.885 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.886 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.902 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=25 2025-11-29 08:41:30.914 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=25 2025-11-29 08:41:30.917 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=28 2025-11-29 08:41:30.926 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=28 2025-11-29 08:41:30.933 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-11-29 08:41:30.935 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-11-29 08:41:30.938 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=17 2025-11-29 08:41:30.944 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-11-29 08:41:30.945 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-11-29 08:41:30.947 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=35 2025-11-29 08:41:30.949 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-11-29 08:41:30.949 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=32 2025-11-29 08:41:30.949 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-11-29 08:41:30.953 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=17 2025-11-29 08:41:30.955 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-11-29 08:41:30.961 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=35 2025-11-29 08:41:30.962 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.963 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-11-29 08:41:30.964 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=32 2025-11-29 08:41:30.965 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-11-29 08:41:30.965 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:41:30.971 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-11-29 08:41:30.976 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:30.978 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:30.982 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:30.988 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:30.988 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:30.989 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:30.991 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:30.995 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:31.003 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=23 2025-11-29 08:41:31.014 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=23 2025-11-29 08:41:31.018 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=26 2025-11-29 08:41:31.027 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-11-29 08:41:31.028 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=26 2025-11-29 08:41:31.032 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:31.033 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=18 2025-11-29 08:41:31.038 INFO 12717 --- [host-check-host-status6,5951d1b5f61fc3d7,c39c65f966bfdc8f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-11-29 08:41:31.040 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-11-29 08:41:31.041 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-11-29 08:41:31.043 INFO 12717 --- [host-check-host-status2,5951d1b5f61fc3d7,abc81d442e363f24] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=18 2025-11-29 08:41:31.044 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-11-29 08:41:31.046 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=36 2025-11-29 08:41:31.049 INFO 12717 --- [host-check-host-status1,5951d1b5f61fc3d7,36c30f1cd3c0229b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-11-29 08:41:31.051 INFO 12717 --- [host-check-host-status3,5951d1b5f61fc3d7,9cd56e9d76bf9465] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-11-29 08:41:31.052 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-11-29 08:41:31.052 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=33 2025-11-29 08:41:31.054 INFO 12717 --- [host-check-host-status4,5951d1b5f61fc3d7,07bf8d1961066fba] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-11-29 08:41:31.054 INFO 12717 --- [host-check-host-status12,5951d1b5f61fc3d7,d6d1ee162feb69e3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=36 2025-11-29 08:41:31.061 INFO 12717 --- [host-check-host-status10,5951d1b5f61fc3d7,7d9782b084d66a45] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=33 2025-11-29 08:41:31.063 INFO 12717 --- [host-check-host-status5,5951d1b5f61fc3d7,080acd5b8c11dd3d] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-11-29 08:41:31.083 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=24 2025-11-29 08:41:31.100 INFO 12717 --- [host-check-host-status7,5951d1b5f61fc3d7,4ffbe6b2eb26481f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=24 2025-11-29 08:41:31.150 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=38 2025-11-29 08:41:31.158 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=38 2025-11-29 08:41:31.227 INFO 12717 --- [host-check-clockdiff10,1517d472912e898b,26ca7ab637f364b8] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376891 0 0 , error= 2025-11-29 08:41:31.234 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=29 2025-11-29 08:41:31.243 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=29 2025-11-29 08:41:31.249 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:31.256 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:31.260 WARN 12717 --- [host-check-clockdiff2,1517d472912e898b,4497cb4278fe4c04] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.50], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:31.260 ERROR 12717 --- [host-check-clockdiff2,1517d472912e898b,4497cb4278fe4c04] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.50] 2025-11-29 08:41:31.260 WARN 12717 --- [host-check-clockdiff2,1517d472912e898b,4497cb4278fe4c04] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.202.22.50, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.50 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:31.315 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=27 2025-11-29 08:41:31.325 INFO 12717 --- [host-check-host-status9,5951d1b5f61fc3d7,0fd27192e3aa7dbf] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=27 2025-11-29 08:41:31.328 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:41:31.462 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:41:31.466 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:41:31.489 INFO 12717 --- [ocp-schedules-6,800cbd605ce92065,734fe04800038c96] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:41:31.490 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=39 2025-11-29 08:41:31.498 WARN 12717 --- [host-check-clockdiff4,1517d472912e898b,879d1b01a984696e] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.52], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:31.498 ERROR 12717 --- [host-check-clockdiff4,1517d472912e898b,879d1b01a984696e] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.52] 2025-11-29 08:41:31.498 WARN 12717 --- [host-check-clockdiff4,1517d472912e898b,879d1b01a984696e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.202.22.52, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.52 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:31.507 WARN 12717 --- [scanner-11213807266-1,800cbd605ce92065,d31aca44cc0b4901] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:41:31.509 INFO 12717 --- [host-check-host-status11,5951d1b5f61fc3d7,909afb6b300cd644] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=39 2025-11-29 08:41:31.606 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=30 2025-11-29 08:41:31.617 INFO 12717 --- [host-check-host-status8,5951d1b5f61fc3d7,48176e790dd8cb61] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=30 2025-11-29 08:41:31.643 INFO 12717 --- [host-check-clockdiff12,1517d472912e898b,847cc08dd9afc6b4] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376891 -1 -2 , error= 2025-11-29 08:41:31.685 INFO 12717 --- [ocp-async-28,6ce185a7272d0a02,7263513a211ab92e] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:31.811 WARN 12717 --- [host-check-clockdiff5,1517d472912e898b,dbe97e6ce86a2825] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.53], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:31.811 ERROR 12717 --- [host-check-clockdiff5,1517d472912e898b,dbe97e6ce86a2825] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.53] 2025-11-29 08:41:31.811 WARN 12717 --- [host-check-clockdiff5,1517d472912e898b,dbe97e6ce86a2825] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.202.22.53, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.53 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:32.220 WARN 12717 --- [host-check-clockdiff3,1517d472912e898b,67aaed037d1163b0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.51], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:32.220 ERROR 12717 --- [host-check-clockdiff3,1517d472912e898b,67aaed037d1163b0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.51] 2025-11-29 08:41:32.220 WARN 12717 --- [host-check-clockdiff3,1517d472912e898b,67aaed037d1163b0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.202.22.51, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.51 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:32.407 INFO 12717 --- [prometheus-minute-schedule-1,7295b31112fff38d,3375784169831555] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376920, interval=60 2025-11-29 08:41:32.414 INFO 12717 --- [prometheus-minute-schedule-1,7295b31112fff38d,3375784169831555] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:41:32.471 WARN 12717 --- [host-check-clockdiff6,1517d472912e898b,72f8663176060cdd] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.54], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:32.471 ERROR 12717 --- [host-check-clockdiff6,1517d472912e898b,72f8663176060cdd] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.54] 2025-11-29 08:41:32.471 WARN 12717 --- [host-check-clockdiff6,1517d472912e898b,72f8663176060cdd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.202.22.54, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.54 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:32.484 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-98,56c9ce12273433eb,946f7c25860f1ea0] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=56c9ce12273433eb, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:41:32.486 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-98,56c9ce12273433eb,946f7c25860f1ea0] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=56c9ce12273433eb, duration=2 ms] 2025-11-29 08:41:32.667 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,7ebf730bc4be40b8,0a30f042b8e677da] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=7ebf730bc4be40b8, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:41:32.667 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,7ebf730bc4be40b8,0a30f042b8e677da] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=7ebf730bc4be40b8, duration=1 ms] 2025-11-29 08:41:32.736 WARN 12717 --- [host-check-clockdiff11,1517d472912e898b,40a065a2c485868b] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.48], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:32.736 ERROR 12717 --- [host-check-clockdiff11,1517d472912e898b,40a065a2c485868b] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.48] 2025-11-29 08:41:32.736 WARN 12717 --- [host-check-clockdiff11,1517d472912e898b,40a065a2c485868b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=12, ip=10.202.22.48, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.48 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:33.128 INFO 12717 --- [ocp-async-1,2d6a0c7fa08486bb,6050b152e18fe046] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:41:33.593 WARN 12717 --- [host-check-clockdiff9,1517d472912e898b,c3ac2b15faf3acf4] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.61], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:33.593 ERROR 12717 --- [host-check-clockdiff9,1517d472912e898b,c3ac2b15faf3acf4] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.61] 2025-11-29 08:41:33.593 WARN 12717 --- [host-check-clockdiff9,1517d472912e898b,c3ac2b15faf3acf4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=10, ip=10.202.22.61, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.61 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:33.685 INFO 12717 --- [ocp-async-23,6e051ef128bcf662,0db672d3689f09de] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:33.717 INFO 12717 --- [ocp-schedules-1,c672de03a1246d09,25e5ef1d99756287] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:41:33.717 INFO 12717 --- [ocp-schedules-1,c672de03a1246d09,25e5ef1d99756287] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=9 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=8 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=12 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-29,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=10 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=11 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=13 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=6 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2025-11-29 08:41:33.717 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2025-11-29 08:41:33.723 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:41:33.724 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:41:33.724 INFO 12717 --- [ForkJoinPool.commonPool-worker-29,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:41:33.724 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:41:33.724 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:41:33.724 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:41:33.724 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:41:33.724 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:41:33.724 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:41:33.724 INFO 12717 --- [ocp-schedules-1,c672de03a1246d09,25e5ef1d99756287] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:41:33.725 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:41:33.725 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:41:33.725 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:41:33.725 INFO 12717 --- [ForkJoinPool.commonPool-worker-29,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:41:33.725 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:41:33.725 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:41:33.726 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:41:33.726 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:41:33.726 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:41:33.726 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:41:33.726 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:41:33.726 INFO 12717 --- [ocp-schedules-1,c672de03a1246d09,25e5ef1d99756287] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:41:33.727 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:41:33.728 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:41:33.751 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.752 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.752 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.752 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.752 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.753 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.753 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.753 INFO 12717 --- [ForkJoinPool.commonPool-worker-29,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.754 INFO 12717 --- [ocp-schedules-1,c672de03a1246d09,25e5ef1d99756287] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.754 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.755 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.757 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62889/api/v1/log/alarms, params:null 2025-11-29 08:41:33.808 INFO 12717 --- [ocp-schedules-1,c672de03a1246d09,25e5ef1d99756287] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms finished. 2025-11-29 08:41:33.927 INFO 12717 --- [ocp-schedules-4,d7ee081eeddcae23,0c5f72379f609d2d] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all tenant information executed completed, allCount=2, successCount=2 2025-11-29 08:41:34.363 WARN 12717 --- [host-check-clockdiff7,1517d472912e898b,8af17ee15ff1e5a7] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.59], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:34.363 ERROR 12717 --- [host-check-clockdiff7,1517d472912e898b,8af17ee15ff1e5a7] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.59] 2025-11-29 08:41:34.363 WARN 12717 --- [host-check-clockdiff7,1517d472912e898b,8af17ee15ff1e5a7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=8, ip=10.202.22.59, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.59 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:41:34.466 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:41:34.468 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:41:34.687 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,e6760d80539fa580,73d3deca1c9643d7] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=e6760d80539fa580, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:41:34.691 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,e6760d80539fa580,73d3deca1c9643d7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=e6760d80539fa580, duration=5 ms] 2025-11-29 08:41:35.180 INFO 12717 --- [ocp-schedules-5,20e78cd03f888d83,48b19db3c427102a] c.o.o.s.c.scheduled.ClusterSchedules : Collect all cluster compaction info. 2025-11-29 08:41:35.182 INFO 12717 --- [ocp-schedules-5,20e78cd03f888d83,48b19db3c427102a] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:41:35.182 INFO 12717 --- [Collect compaction for all clusters1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-11-29 08:41:35.182 INFO 12717 --- [Collect compaction for all clusters2,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-11-29 08:41:35.407 INFO 12717 --- [prometheus-minute-schedule-1,a3fb34ce7cca3f16,e63b38336ba4f42f] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376920, interval=60 2025-11-29 08:41:35.417 INFO 12717 --- [prometheus-minute-schedule-1,a3fb34ce7cca3f16,e63b38336ba4f42f] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:41:35.439 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,fe2302cffe26c3c2,24ab836d76e6ed32] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=fe2302cffe26c3c2, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:41:35.439 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,fe2302cffe26c3c2,24ab836d76e6ed32] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=fe2302cffe26c3c2, duration=1 ms] 2025-11-29 08:41:35.685 INFO 12717 --- [ocp-async-23,5a4a7942e1ad7e54,448cf73ebe525e81] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:35.740 INFO 12717 --- [ocp-schedules-26,8a995b4a66ff63f7,3f436ef7bd8a5c05] .o.o.s.t.b.c.AbstractShardingClusterTask : Refresh root service list executed completed, allCount=2, successCount=2 2025-11-29 08:41:35.955 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-89,fd85f597b6ebcd81,a5863940d676134c] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=fd85f597b6ebcd81, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:41:35.955 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-89,fd85f597b6ebcd81,a5863940d676134c] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=fd85f597b6ebcd81, duration=0 ms] 2025-11-29 08:41:36.823 INFO 12717 --- [ocp-schedules-27,6712b561473c7844,0528d881f3304a60] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:41:36.827 INFO 12717 --- [ocp-schedules-27,6712b561473c7844,0528d881f3304a60] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:41:37.468 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:41:37.470 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:41:37.685 INFO 12717 --- [ocp-async-7,28ab0d35ca6258c3,6acccc9316c81eed] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:38.407 INFO 12717 --- [prometheus-minute-schedule-1,f7f211881e7fe328,d8710910489ff555] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376920, interval=60 2025-11-29 08:41:38.413 INFO 12717 --- [prometheus-minute-schedule-1,f7f211881e7fe328,d8710910489ff555] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:41:39.276 INFO 12717 --- [ocp-schedules-20,42eb5b7f2d0d7e50,8d680dc7f681be55] c.o.o.s.common.scheduled.PerfSchedules : Collect slow sql for alarm, time=2025-11-29T08:41:39.276+08:00, clusters=[1, 2] 2025-11-29 08:41:39.283 INFO 12717 --- [ocp-schedules-20,42eb5b7f2d0d7e50,8d680dc7f681be55] c.o.o.p.s.i.alarm.SlowSqlDataSource : query sqlCount of obCluster ocpserver for slowSqlStat, sql count is 0. 2025-11-29 08:41:39.369 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.o.s.c.scheduled.TenantSchedules : Check suspend tenant compaction status 2025-11-29 08:41:39.369 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] s.t.b.t.CheckSuspendTenantCompactionTask : Check tenant compaction status. 2025-11-29 08:41:39.408 INFO 12717 --- [ocp-schedules-20,42eb5b7f2d0d7e50,8d680dc7f681be55] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for DB, save 20 to SqlAggregateData 2025-11-29 08:41:39.422 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:39.423 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-11-29 08:41:39.445 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:41:39.445 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:41:39.445 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:41:39.445 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:41:39.458 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:41:39.458 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:41:39.458 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:41:39.458 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:41:39.458 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:39.459 INFO 12717 --- [ocp-schedules-29,14d988e1e5bb2a80,abc325399b74f0b1] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-11-29 08:41:39.672 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:41:39.671911+08:00, lastProcessTime=2025-11-29T08:41:29.685821+08:00 2025-11-29 08:41:39.685 INFO 12717 --- [ocp-async-7,f9d75854b0dc7cd3,6acfabf70bf0698f] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:40.013 INFO 12717 --- [ocp-schedules-19,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=3 2025-11-29 08:41:40.013 INFO 12717 --- [ocp-schedules-23,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=3 2025-11-29 08:41:40.024 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:41:39.671911+08:00 2025-11-29 08:41:40.040 INFO 12717 --- [Check obproxy1,516694c2d6ba3c65,38cd61b88c818576] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:41:40.040 INFO 12717 --- [Check obproxy1,516694c2d6ba3c65,38cd61b88c818576] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:41:40.040 INFO 12717 --- [Collect obproxy1,ba359c85db1e1f69,1850e2122da357d8] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:41:40.040 INFO 12717 --- [Check obproxy3,4f377dd41b9504c1,6d4c2a5360e42673] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:41:40.040 INFO 12717 --- [Check obproxy2,8a7cfff42a50fa3b,598801e2158c31d7] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:41:40.040 INFO 12717 --- [Check obproxy3,4f377dd41b9504c1,6d4c2a5360e42673] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:41:40.040 INFO 12717 --- [Check obproxy2,8a7cfff42a50fa3b,598801e2158c31d7] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:41:40.040 INFO 12717 --- [Collect obproxy1,ba359c85db1e1f69,1850e2122da357d8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:41:40.041 INFO 12717 --- [Collect obproxy2,283316971d79b416,2f7e919156e86c65] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:41:40.041 INFO 12717 --- [Collect obproxy2,283316971d79b416,2f7e919156e86c65] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:41:40.051 INFO 12717 --- [Collect obproxy3,0e35ba7af828a73d,8b2485d65032b225] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:41:40.051 INFO 12717 --- [Collect obproxy3,0e35ba7af828a73d,8b2485d65032b225] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:41:40.077 INFO 12717 --- [Collect obproxy2,283316971d79b416,2f7e919156e86c65] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:41:40.078 INFO 12717 --- [Check obproxy3,4f377dd41b9504c1,6d4c2a5360e42673] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:41:40.081 INFO 12717 --- [Check obproxy3,4f377dd41b9504c1,6d4c2a5360e42673] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2025-11-29 08:41:40.081 INFO 12717 --- [Check obproxy3,4f377dd41b9504c1,6d4c2a5360e42673] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 3 proxy_id: 3, need to sync to meta: false 2025-11-29 08:41:40.082 INFO 12717 --- [Collect obproxy2,283316971d79b416,2f7e919156e86c65] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:41:40.086 INFO 12717 --- [Check obproxy2,8a7cfff42a50fa3b,598801e2158c31d7] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:41:40.086 INFO 12717 --- [Collect obproxy2,283316971d79b416,2f7e919156e86c65] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:41:40.090 INFO 12717 --- [Check obproxy1,516694c2d6ba3c65,38cd61b88c818576] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:41:40.091 INFO 12717 --- [Collect obproxy1,ba359c85db1e1f69,1850e2122da357d8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:41:40.092 INFO 12717 --- [Check obproxy2,8a7cfff42a50fa3b,598801e2158c31d7] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-11-29 08:41:40.092 INFO 12717 --- [Check obproxy2,8a7cfff42a50fa3b,598801e2158c31d7] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 2, need to sync to meta: false 2025-11-29 08:41:40.095 INFO 12717 --- [Check obproxy1,516694c2d6ba3c65,38cd61b88c818576] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-11-29 08:41:40.095 INFO 12717 --- [Check obproxy1,516694c2d6ba3c65,38cd61b88c818576] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 1, need to sync to meta: false 2025-11-29 08:41:40.097 INFO 12717 --- [Collect obproxy1,ba359c85db1e1f69,1850e2122da357d8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:41:40.102 INFO 12717 --- [Collect obproxy1,ba359c85db1e1f69,1850e2122da357d8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:41:40.102 INFO 12717 --- [Collect obproxy3,0e35ba7af828a73d,8b2485d65032b225] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:41:40.106 INFO 12717 --- [Collect obproxy3,0e35ba7af828a73d,8b2485d65032b225] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:41:40.110 INFO 12717 --- [Collect obproxy3,0e35ba7af828a73d,8b2485d65032b225] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:41:40.147 INFO 12717 --- [ocp-schedules-21,fc256618889eb0a8,a351313c6c267b00] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:41:40.175 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-91,6e6fe6d6a31cfe41,0a5fa4d0f57e2ff2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=6e6fe6d6a31cfe41, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:41:40.175 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-91,6e6fe6d6a31cfe41,0a5fa4d0f57e2ff2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=6e6fe6d6a31cfe41, duration=1 ms] 2025-11-29 08:41:40.275 INFO 12717 --- [ocp-schedules-22,fbb4322532a2423b,6ca88007f4f0824d] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=2, successCount=2 2025-11-29 08:41:40.470 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:41:40.471 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:41:40.482 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-87,38e43cfdec18c483,2108171a9aa3615b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=38e43cfdec18c483, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:41:40.484 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-87,38e43cfdec18c483,2108171a9aa3615b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=38e43cfdec18c483, duration=2 ms] 2025-11-29 08:41:40.588 INFO 12717 --- [ocp-schedules-13,f1da2dc9ce2892c1,4ba22bffaf54ec16] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:41:41.407 INFO 12717 --- [prometheus-minute-schedule-1,31634083adc90382,e058a99e00716b72] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376920, interval=60 2025-11-29 08:41:41.414 INFO 12717 --- [prometheus-minute-schedule-1,31634083adc90382,e058a99e00716b72] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:41:41.624 INFO 12717 --- [ocp-schedules-16,1c63e513882c3acc,41a26ca334b3a1f5] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:41:41.630 INFO 12717 --- [alarm-detect-5,1c63e513882c3acc,7e2b6cc6ddd20afc] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008286, data is empty. 2025-11-29 08:41:41.631 INFO 12717 --- [alarm-detect-8,1c63e513882c3acc,18cc165efccc71a7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008286, data is empty. 2025-11-29 08:41:41.632 INFO 12717 --- [alarm-detect-6,1c63e513882c3acc,73128408057bb6a7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008286, data is empty. 2025-11-29 08:41:41.636 INFO 12717 --- [alarm-detect-6,1c63e513882c3acc,73128408057bb6a7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008306, data is empty. 2025-11-29 08:41:41.637 INFO 12717 --- [alarm-detect-5,1c63e513882c3acc,7e2b6cc6ddd20afc] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008306, data is empty. 2025-11-29 08:41:41.637 INFO 12717 --- [alarm-detect-8,1c63e513882c3acc,18cc165efccc71a7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008306, data is empty. 2025-11-29 08:41:41.644 WARN 12717 --- [scanner-11213807494-1,1c63e513882c3acc,736b284ffc15a65e] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:41:41.686 INFO 12717 --- [ocp-async-1,6b38131ec232c77b,2bd989ebcd8ad145] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:41.701 INFO 12717 --- [alarm-detect-9,1c63e513882c3acc,a180ced844638c92] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008286, data is empty. 2025-11-29 08:41:41.705 INFO 12717 --- [alarm-detect-9,1c63e513882c3acc,a180ced844638c92] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008306, data is empty. 2025-11-29 08:41:41.963 INFO 12717 --- [host-check-clockdiff8,1517d472912e898b,47c6ad3e1f3b7626] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376901 -5 -6 , error= 2025-11-29 08:41:43.128 INFO 12717 --- [ocp-async-36,5664e16f1bac52f1,139cfd6a27de7873] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:41:43.450 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,e7e1d0f08709d9f0,e2c9cad67453f179] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=e7e1d0f08709d9f0, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:41:43.451 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,e7e1d0f08709d9f0,e2c9cad67453f179] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=e7e1d0f08709d9f0, duration=2 ms] 2025-11-29 08:41:43.471 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:41:43.472 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:41:43.686 INFO 12717 --- [ocp-async-28,46286ebdc2f71fc9,2c8849dad7d4b094] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:43.735 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,c4fa94a0e2cadf52,1ca9d7378425db44] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=c4fa94a0e2cadf52, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:41:43.735 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,c4fa94a0e2cadf52,1ca9d7378425db44] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=c4fa94a0e2cadf52, duration=1 ms] 2025-11-29 08:41:44.407 INFO 12717 --- [prometheus-minute-schedule-1,c4b234ba01ee9427,98a466ed799e56f6] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376920, interval=60 2025-11-29 08:41:44.414 INFO 12717 --- [prometheus-minute-schedule-1,c4b234ba01ee9427,98a466ed799e56f6] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:41:44.746 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,4fc8b188456b3ad8,9eb40c86ff182f78] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=4fc8b188456b3ad8, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:41:44.747 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,4fc8b188456b3ad8,9eb40c86ff182f78] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=4fc8b188456b3ad8, duration=1 ms] 2025-11-29 08:41:45.182 INFO 12717 --- [ocp-schedules-5,20e78cd03f888d83,48b19db3c427102a] .o.o.s.t.b.c.AbstractShardingClusterTask : Collect compaction for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:41:45.686 INFO 12717 --- [ocp-async-37,6a71ee2b815d9667,8ccea9be3894bbe1] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:46.281 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,853b44fd0be65803,3c07b03b0325f40a] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=853b44fd0be65803, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:41:46.283 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,853b44fd0be65803,3c07b03b0325f40a] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=853b44fd0be65803, duration=2 ms] 2025-11-29 08:41:46.455 INFO 12717 --- [ocp-schedules-4,2e527ea8d0a80b96,077e87084705d866] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-11-29 08:41:46.472 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:41:46.472 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:41:46.582 INFO 12717 --- [ocp-schedules-4,2e527ea8d0a80b96,077e87084705d866] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:41:46.582 INFO 12717 --- [ocp-schedules-4,2e527ea8d0a80b96,077e87084705d866] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:41:46.945 INFO 12717 --- [ocp-schedules-1,13b3b8082bca69a5,c2649bc5ea7672a4] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:41:46.948 INFO 12717 --- [ocp-schedules-1,13b3b8082bca69a5,c2649bc5ea7672a4] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:41:47.135 INFO 12717 --- [ocp-schedules-14,74cff69c4030e2df,f971439fbe7e9345] c.o.o.s.c.scheduled.TenantSchedules : Collect all tenant major compaction 2025-11-29 08:41:47.135 INFO 12717 --- [ocp-schedules-14,74cff69c4030e2df,f971439fbe7e9345] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2025-11-29 08:41:47.155 INFO 12717 --- [Collect all tenants major compactions2,74cff69c4030e2df,04aba4bf072e93ac] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:ocp_monitor 2025-11-29 08:41:47.156 INFO 12717 --- [Collect all tenants major compactions1,74cff69c4030e2df,3366b76e5cc814dd] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:ocp_meta 2025-11-29 08:41:47.156 INFO 12717 --- [Collect all tenants major compactions3,74cff69c4030e2df,83dc16dad9bce921] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:sys 2025-11-29 08:41:47.156 INFO 12717 --- [Collect all tenants major compactions6,74cff69c4030e2df,5e43b0a728738a2e] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:oms 2025-11-29 08:41:47.156 INFO 12717 --- [Collect all tenants major compactions5,74cff69c4030e2df,1e9d5b26e7f0ac40] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:nextcloud 2025-11-29 08:41:47.156 INFO 12717 --- [Collect all tenants major compactions4,74cff69c4030e2df,e5dfa93be0029028] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:lhp6796 2025-11-29 08:41:47.156 INFO 12717 --- [Collect all tenants major compactions7,74cff69c4030e2df,39967c3bca4d9e96] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:pipeline 2025-11-29 08:41:47.157 INFO 12717 --- [Collect all tenants major compactions8,74cff69c4030e2df,aa7c65e8e3daa246] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:sys 2025-11-29 08:41:47.158 INFO 12717 --- [Collect all tenants major compactions10,74cff69c4030e2df,6bee57f78e8708aa] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:zto 2025-11-29 08:41:47.158 INFO 12717 --- [Collect all tenants major compactions9,74cff69c4030e2df,d18308954851d958] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:wecom_bot 2025-11-29 08:41:47.168 INFO 12717 --- [Collect all tenants major compactions3,74cff69c4030e2df,83dc16dad9bce921] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:47.168 INFO 12717 --- [Collect all tenants major compactions1,74cff69c4030e2df,3366b76e5cc814dd] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:47.169 INFO 12717 --- [Collect all tenants major compactions2,74cff69c4030e2df,04aba4bf072e93ac] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:47.170 INFO 12717 --- [Collect all tenants major compactions3,74cff69c4030e2df,83dc16dad9bce921] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-11-29 08:41:47.170 INFO 12717 --- [Collect all tenants major compactions1,74cff69c4030e2df,3366b76e5cc814dd] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2025-11-29 08:41:47.171 INFO 12717 --- [Collect all tenants major compactions2,74cff69c4030e2df,04aba4bf072e93ac] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1004] 2025-11-29 08:41:47.174 INFO 12717 --- [Collect all tenants major compactions1,74cff69c4030e2df,3366b76e5cc814dd] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:ocp_meta 2025-11-29 08:41:47.174 INFO 12717 --- [Collect all tenants major compactions3,74cff69c4030e2df,83dc16dad9bce921] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:sys 2025-11-29 08:41:47.175 INFO 12717 --- [Collect all tenants major compactions2,74cff69c4030e2df,04aba4bf072e93ac] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:ocp_monitor 2025-11-29 08:41:47.177 INFO 12717 --- [Collect all tenants major compactions4,74cff69c4030e2df,e5dfa93be0029028] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:47.177 INFO 12717 --- [Collect all tenants major compactions5,74cff69c4030e2df,1e9d5b26e7f0ac40] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:47.177 INFO 12717 --- [Collect all tenants major compactions9,74cff69c4030e2df,d18308954851d958] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:47.177 INFO 12717 --- [Collect all tenants major compactions8,74cff69c4030e2df,aa7c65e8e3daa246] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:47.178 INFO 12717 --- [Collect all tenants major compactions7,74cff69c4030e2df,39967c3bca4d9e96] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:47.178 INFO 12717 --- [Collect all tenants major compactions6,74cff69c4030e2df,5e43b0a728738a2e] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:47.178 INFO 12717 --- [Collect all tenants major compactions5,74cff69c4030e2df,1e9d5b26e7f0ac40] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1018] 2025-11-29 08:41:47.178 INFO 12717 --- [Collect all tenants major compactions4,74cff69c4030e2df,e5dfa93be0029028] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1010] 2025-11-29 08:41:47.179 INFO 12717 --- [Collect all tenants major compactions10,74cff69c4030e2df,6bee57f78e8708aa] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:47.179 INFO 12717 --- [Collect all tenants major compactions9,74cff69c4030e2df,d18308954851d958] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1024] 2025-11-29 08:41:47.179 INFO 12717 --- [Collect all tenants major compactions8,74cff69c4030e2df,aa7c65e8e3daa246] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-11-29 08:41:47.180 INFO 12717 --- [Collect all tenants major compactions7,74cff69c4030e2df,39967c3bca4d9e96] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1020] 2025-11-29 08:41:47.180 INFO 12717 --- [Collect all tenants major compactions6,74cff69c4030e2df,5e43b0a728738a2e] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1014] 2025-11-29 08:41:47.180 INFO 12717 --- [Collect all tenants major compactions10,74cff69c4030e2df,6bee57f78e8708aa] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1022] 2025-11-29 08:41:47.182 INFO 12717 --- [Collect all tenants major compactions7,74cff69c4030e2df,39967c3bca4d9e96] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:pipeline 2025-11-29 08:41:47.183 INFO 12717 --- [Collect all tenants major compactions8,74cff69c4030e2df,aa7c65e8e3daa246] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:sys 2025-11-29 08:41:47.184 INFO 12717 --- [Collect all tenants major compactions9,74cff69c4030e2df,d18308954851d958] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:wecom_bot 2025-11-29 08:41:47.184 INFO 12717 --- [Collect all tenants major compactions10,74cff69c4030e2df,6bee57f78e8708aa] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:zto 2025-11-29 08:41:47.185 INFO 12717 --- [Collect all tenants major compactions5,74cff69c4030e2df,1e9d5b26e7f0ac40] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:nextcloud 2025-11-29 08:41:47.185 INFO 12717 --- [Collect all tenants major compactions6,74cff69c4030e2df,5e43b0a728738a2e] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:oms 2025-11-29 08:41:47.186 INFO 12717 --- [Collect all tenants major compactions4,74cff69c4030e2df,e5dfa93be0029028] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:lhp6796 2025-11-29 08:41:47.187 INFO 12717 --- [ocp-schedules-14,74cff69c4030e2df,f971439fbe7e9345] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2025-11-29 08:41:47.407 INFO 12717 --- [prometheus-minute-schedule-1,29900f2a2c72155a,f6ad118cd6c6aca5] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376920, interval=60 2025-11-29 08:41:47.414 INFO 12717 --- [prometheus-minute-schedule-1,29900f2a2c72155a,f6ad118cd6c6aca5] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:41:47.682 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,0bacce97a19f6972,670068fc88029437] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=0bacce97a19f6972, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:41:47.682 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,0bacce97a19f6972,670068fc88029437] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=0bacce97a19f6972, duration=1 ms] 2025-11-29 08:41:47.687 INFO 12717 --- [ocp-async-37,a99b82b227ba57ef,a8ae2fb6956b8738] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:48.902 INFO 12717 --- [ocp-schedules-4,3a318dbd9a19ba6c,33204e6ea5566693] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-11-29 08:41:49.408 INFO 12717 --- [prometheus-cache-monitor1,ad0bce1a732c43ab,d8d3dc077cf38e51] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":40356,"minuteWriteQueue":0,"minuteReadCacheSize":17215,"secondWriteCacheSize":10089,"seriesIdKeySize":17387,"secondWriteQueue":0,"minuteWriteCacheSize":17216} 2025-11-29 08:41:49.408 INFO 12717 --- [prometheus-cache-monitor1,ad0bce1a732c43ab,d8d3dc077cf38e51] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{ob_sysstat_40012=49, ob_server_resource_memory_assigned_bytes=23, ob_sysstat_40011=49, ob_sysstat_40010=49, ob_unit_num=23, ob_database_disk_required_size=74, go_memstats_mspan_sys_bytes=25, ob_sysstat_30213=42, go_memstats_stack_sys_bytes=25, ob_sysstat_30212=42, ob_sysstat_30217=42, ob_sysstat_30216=42, ob_tenant_request_queue_total_size=72, jvm_gc_memory_allocated_bytes_total=2, ob_system_event_total_waits=59, ob_tenant_disk_v4_log_disk_size=40, ob_sysstat_40019=49, ob_sysstat_40018=49, processor_process_metrics_total=30, jvm_gc_max_data_size_bytes=2, go_memstats_frees_total=25, host_available=29, ob_sysstat_30220=42, ob_sysstat_30222=42, ob_sysstat_30221=42, mysql_output_write_sql_failed_count=29, ob_database_disk_data_size=73, ob_tenant_memstore_freeze_cnt=40, ob_tenant_unit_min_memory=96, ob_sysstat_60087=49, ob_memstore_total_bytes=72, jvm_threads_states_threads=8, process_resident_memory_bytes=25, ob_cache_size_bytes=67, ob_tenant_unit_max_memory=96, input_collect_metrics_total=34, ob_sysstat_50039=50, ob_sysstat_60083=50, ob_sysstat_50038=49, node_disk_io_time_weighted_seconds_total=39, process_open_fds=25, ob_sysstat_50037=49, process_memory_rss_bytes=41, ob_server_resource_cpu_assigned_percent=23, ob_sysstat_50030=50, sql_audit_input_collect_lost_total=36, go_memstats_heap_inuse_bytes=25, ob_sysstat_40118=49, ob_sysstat_40117=49, node_memory_memfree_bytes=32, ob_sysstat_40116=49, ob_sysstat_40005=49, process_exists=50, ob_sysstat_40004=49, ob_sysstat_40003=49, ob_sysstat_40002=49, go_gc_duration_seconds=30, ob_server_resource_disk_bytes=23, ob_sysstat_40001=49, ob_sysstat_40000=49, mysql_output_write_sql_time=33, ob_sysstat_30200=42, odp_current_session=36, ob_sysstat_30201=42, ob_sysstat_30204=42, ob_sysstat_30205=42, ob_sysstat_30208=42, process_thread_count=41, ob_sysstat_30209=42, ob_sysstat_50040=50, ob_sysstat_40009=49, ob_disk_free_bytes=23, ob_sysstat_40008=49, node_filesystem_avail_bytes=67, ob_sysstat_40007=49, ob_sysstat_40006=49, node_time_seconds=31, ob_unit_config_max_i 2025-11-29 08:41:49.473 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:41:49.473 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:41:49.650 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:41:49.649946+08:00, lastProcessTime=2025-11-29T08:41:39.671911+08:00 2025-11-29 08:41:49.687 INFO 12717 --- [ocp-async-33,3aa3d17737d0d6ab,000b803e5bc9ade5] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:49.875 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:41:49.649946+08:00 2025-11-29 08:41:50.153 INFO 12717 --- [ocp-schedules-26,2ae789909b69c4cf,5d244f81d810b153] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:41:50.407 INFO 12717 --- [prometheus-minute-schedule-1,78b192a21ae95dc9,8de23870ffcc5a7d] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376920, interval=60 2025-11-29 08:41:50.413 INFO 12717 --- [prometheus-minute-schedule-1,78b192a21ae95dc9,8de23870ffcc5a7d] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:41:51.263 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,cb712e69772a50b4,b2deab23a8f08bc2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=cb712e69772a50b4, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:41:51.265 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,cb712e69772a50b4,b2deab23a8f08bc2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=cb712e69772a50b4, duration=3 ms] 2025-11-29 08:41:51.687 INFO 12717 --- [ocp-async-6,369cf5bd502025e1,5180115a1ffbacb5] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:51.734 INFO 12717 --- [ocp-schedules-21,2bcd9d71389e0d7c,07067f3974e0da6f] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:41:51.751 WARN 12717 --- [scanner-11213807704-1,2bcd9d71389e0d7c,586ad73bc99dd90d] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:41:52.032 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,9685cd34ce31d391,9b9e26315f0a2d31] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=9685cd34ce31d391, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:41:52.032 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,9685cd34ce31d391,9b9e26315f0a2d31] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=9685cd34ce31d391, duration=1 ms] 2025-11-29 08:41:52.473 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:41:52.474 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:41:53.128 INFO 12717 --- [ocp-async-25,0227f62511f8364f,a5406f6ce051e275] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:41:53.407 INFO 12717 --- [prometheus-minute-schedule-1,7c05c8ba2218fdca,253b6f75c8119f4a] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376920, interval=60 2025-11-29 08:41:53.413 INFO 12717 --- [prometheus-minute-schedule-1,7c05c8ba2218fdca,253b6f75c8119f4a] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:41:53.539 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,ba86936e3a109360,bd29584ace9b2be4] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=ba86936e3a109360, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:41:53.539 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,ba86936e3a109360,bd29584ace9b2be4] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=ba86936e3a109360, duration=1 ms] 2025-11-29 08:41:53.687 INFO 12717 --- [ocp-async-32,5338dfa847a92996,1d52d5caa8bdb7f4] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:54.095 INFO 12717 --- [ocp-schedules-1,c96f0a2bd169450f,1e6cff06138d3f1b] c.o.o.alarm.service.OcpAlarmServiceImpl : fix alarm target, alarms size 48 2025-11-29 08:41:54.098 INFO 12717 --- [ocp-schedules-12,f879bfe0bdcc9ce6,e4025d12f44f0c72] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:41:54.098 INFO 12717 --- [ocp-schedules-12,6a7f1a83ea7d2cdc,97f399254c771108] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-11-29 08:41:54.099 INFO 12717 --- [ocp-schedules-12,6a7f1a83ea7d2cdc,97f399254c771108] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:41:54.125 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.125 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:41:54.136 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.137 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:41:54.162 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:41:54.163 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.163 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:41:54.169 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,fe1cad9024cac9e3,539d838971a9cb94] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=fe1cad9024cac9e3, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:41:54.170 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,fe1cad9024cac9e3,539d838971a9cb94] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=fe1cad9024cac9e3, duration=2 ms] 2025-11-29 08:41:54.215 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:41:54.216 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.217 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:41:54.218 WARN 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=OCP 2025-11-29 08:41:54.219 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.222 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:41:54.223 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:41:54.236 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.236 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.329 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:41:54.331 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.331 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:41:54.445 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.456 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:41:54.456 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.457 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:41:54.458 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster ocpserver 2025-11-29 08:41:54.474 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:41:54.475 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.476 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:41:54.479 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.482 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:41:54.486 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:41:54.487 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2025-11-29 08:41:54.492 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:41:54.498 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:41:54.500 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.500 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.516 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.516 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:12+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.532 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.532 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.546 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.546 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:41:54.559 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.559 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.575 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.575 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.590 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.591 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.601 INFO 12717 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:41:54.698 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=8073, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:41:54.707 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:41:54.708 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.708 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:41:54.717 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=ocpserver, obClusterId=1743215593 2025-11-29 08:41:54.717 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=ocpserver, obClusterId=1743215593, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"ocpserver","ObClusterId":1743215593,"ObRegion":"ocpserver","ObRegionId":1743215593,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.58:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376673675000}, createTime=2025-03-29T10:43:06+08:00, creator=admin, obVersion=4.2.1.8, packageOperatingSystem=null, loadType=null, tenantCount=3) 2025-11-29 08:41:54.812 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.812 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:30+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.829 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.829 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.847 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.847 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:41:54.861 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.861 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.877 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.877 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:42+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.896 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.896 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:41:54.914 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:41:54.928 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:41:54.929 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:54.929 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:41:55.112 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster DB 2025-11-29 08:41:55.116 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 2025-11-29 08:41:55.120 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:41:55.122 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:41:55.138 INFO 12717 --- [Sync all clusters2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:41:55.201 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,7f9472619324be6c,636baaee49730df7] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=7f9472619324be6c, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:41:55.202 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,7f9472619324be6c,636baaee49730df7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=7f9472619324be6c, duration=1 ms] 2025-11-29 08:41:55.328 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 got ObProcessInfoResult(pid=2447, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:41:55.343 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:41:55.343 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:41:55.344 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:41:55.361 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=DB, obClusterId=1743215594 2025-11-29 08:41:55.361 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=DB, obClusterId=1743215594, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=CREATED, syncStatus=VALID, rootserverJson={"ObCluster":"DB","ObClusterId":1743215594,"ObRegion":"DB","ObRegionId":1743215594,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.52:2882","role":"LEADER","sql_port":2881},{"address":"10.202.22.49:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.202.22.61:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376629943000}, createTime=2025-03-29T11:15:04+08:00, creator=admin, obVersion=4.3.5.1, packageOperatingSystem=el8, loadType=HTAP, tenantCount=7) 2025-11-29 08:41:55.474 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:41:55.476 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:41:55.687 INFO 12717 --- [ocp-async-38,5c39b69a3b1b1f51,8404a6b54996981e] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:56.407 INFO 12717 --- [prometheus-minute-schedule-1,6a08637363b0f190,3d480babd73a589d] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376920, interval=60 2025-11-29 08:41:56.413 INFO 12717 --- [prometheus-minute-schedule-1,6a08637363b0f190,3d480babd73a589d] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:41:56.995 INFO 12717 --- [ocp-schedules-18,7e4d7f10d642fad4,4fadc266bee466d4] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:41:56.998 INFO 12717 --- [ocp-schedules-18,7e4d7f10d642fad4,4fadc266bee466d4] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:41:57.149 INFO 12717 --- [ocp-schedules-24,025c467bfc8f5b6b,f25c0be2381e3a0b] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2025-11-29 08:41:57.149 INFO 12717 --- [ocp-schedules-24,025c467bfc8f5b6b,f25c0be2381e3a0b] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=817192494921281536 2025-11-29 08:41:57.150 INFO 12717 --- [ocp-schedules-24,025c467bfc8f5b6b,f25c0be2381e3a0b] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2025-11-29 08:41:57.150 INFO 12717 --- [ocp-schedules-24,025c467bfc8f5b6b,f25c0be2381e3a0b] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=851074473877958656 2025-11-29 08:41:57.151 INFO 12717 --- [ocp-schedules-24,025c467bfc8f5b6b,f25c0be2381e3a0b] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2025-11-29 08:41:57.151 INFO 12717 --- [ocp-schedules-24,025c467bfc8f5b6b,f25c0be2381e3a0b] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=834880106968510464 2025-11-29 08:41:57.687 INFO 12717 --- [ocp-async-38,75adbd713715b5f6,13b93d1cbf390221] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:57.826 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,0dd2d0e025e504a6,cb74584eccf7fb9e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=0dd2d0e025e504a6, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:41:57.830 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,0dd2d0e025e504a6,cb74584eccf7fb9e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=0dd2d0e025e504a6, duration=5 ms] 2025-11-29 08:41:58.477 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:41:58.477 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:41:59.407 INFO 12717 --- [prometheus-minute-schedule-1,d20b8f0d79ccd048,5fe9a0e8dcc263f5] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376920, interval=60 2025-11-29 08:41:59.413 INFO 12717 --- [prometheus-minute-schedule-1,d20b8f0d79ccd048,5fe9a0e8dcc263f5] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:41:59.679 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:41:59.678702+08:00, lastProcessTime=2025-11-29T08:41:49.649946+08:00 2025-11-29 08:41:59.687 INFO 12717 --- [ocp-async-35,8e7892eefbfc044b,0d3a68781857304e] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:41:59.915 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:41:59.678702+08:00 2025-11-29 08:42:00.157 INFO 12717 --- [ocp-schedules-14,7865e44b2922fad6,77bb3454e7d0a779] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:42:00.266 INFO 12717 --- [ocp-schedules-5,cd091ba126f09b2a,fc6ba303214c4e8e] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=BackupAlarmCheckSchedule, method=run 2025-11-29 08:42:00.271 INFO 12717 --- [ocp-schedules-2,9f35dd80a6fbeaf8,333a81d03aa05ce8] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-11-29 08:42:00.271 INFO 12717 --- [ocp-schedules-26,de9f021bfe81dbc0,40971b4ec3c02992] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-11-29 08:42:00.271 INFO 12717 --- [ocp-schedules-16,f18446829ea3ff41,40dbf9c83430f0dd] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2025-11-29 08:42:00.271 INFO 12717 --- [ocp-schedules-22,1377a7a917c32ed5,db218fa46084886a] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-11-29 08:42:00.274 INFO 12717 --- [ocp-schedules-26,de9f021bfe81dbc0,40971b4ec3c02992] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:42:00.275 INFO 12717 --- [ocp-schedules-22,1377a7a917c32ed5,db218fa46084886a] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:42:00.282 INFO 12717 --- [ocp-schedules-5,cd091ba126f09b2a,fc6ba303214c4e8e] c.o.o.b.i.alarm.BackupAlarmService : Backup alarm check schedule. 2025-11-29 08:42:00.302 INFO 12717 --- [ocp-schedules-16,f18446829ea3ff41,40dbf9c83430f0dd] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2025-11-29 08:42:00.303 INFO 12717 --- [ocp-schedules-18,f34d03ce53488cad,17deca68aaf7f109] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:42:00.304 INFO 12717 --- [host-check-agent-status1,f34d03ce53488cad,9ffae59262b6f892] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:42:00.304 INFO 12717 --- [host-check-agent-status2,f34d03ce53488cad,1251e06f40444bcf] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:42:00.304 INFO 12717 --- [host-check-agent-status3,f34d03ce53488cad,5cb52acbe33e498a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:42:00.304 INFO 12717 --- [host-check-agent-status4,f34d03ce53488cad,bd253c33dd971170] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:42:00.305 INFO 12717 --- [host-check-agent-status5,f34d03ce53488cad,aab191bae4de797b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:42:00.305 INFO 12717 --- [host-check-agent-status6,f34d03ce53488cad,f885af62d95080ef] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:42:00.305 INFO 12717 --- [host-check-agent-status7,f34d03ce53488cad,848b01ed731d26ef] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:42:00.305 INFO 12717 --- [host-check-agent-status8,f34d03ce53488cad,4e2711287d5f7e40] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:42:00.306 INFO 12717 --- [host-check-agent-status10,f34d03ce53488cad,de7e39a9d1b5a613] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:42:00.306 INFO 12717 --- [ocp-schedules-7,9cfa4a72abdc0490,d81be7b001526c3a] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-11-29 08:42:00.306 INFO 12717 --- [host-check-agent-status9,f34d03ce53488cad,c6a0d09e2fb7be58] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:42:00.306 INFO 12717 --- [host-check-agent-status11,f34d03ce53488cad,d18e250431584051] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:42:00.307 INFO 12717 --- [host-check-agent-status12,f34d03ce53488cad,db67e7f12af22a1e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:42:00.318 INFO 12717 --- [ocp-schedules-8,37a479e416bb1651,adea8c130ec1f184] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:42:00.319 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:42:00.319 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:42:00.319 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:42:00.320 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:42:00.320 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:42:00.320 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:42:00.320 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:42:00.321 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:42:00.321 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:42:00.321 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:42:00.321 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:42:00.322 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:42:00.335 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,77ffa0491862f2db,fa44df62f06b293b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=77ffa0491862f2db, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:00.336 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,77ffa0491862f2db,fa44df62f06b293b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=77ffa0491862f2db, duration=1 ms] 2025-11-29 08:42:00.338 INFO 12717 --- [ocp-schedules-11,8c7a43e4c5067e23,e18b447dc5e988fc] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster root server. 2025-11-29 08:42:00.353 INFO 12717 --- [ocp-schedules-11,8c7a43e4c5067e23,e18b447dc5e988fc] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:42:00.353 INFO 12717 --- [ocp-schedules-1,f9f61431ecd5626e,e76d6af26d959b24] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:42:00.354 INFO 12717 --- [host-check-clockdiff3,f9f61431ecd5626e,231f786dd9227d05] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.51], timeoutMillis=20000 2025-11-29 08:42:00.354 INFO 12717 --- [host-check-clockdiff1,f9f61431ecd5626e,49fcd7b1b0840134] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.58], timeoutMillis=20000 2025-11-29 08:42:00.354 INFO 12717 --- [host-check-clockdiff2,f9f61431ecd5626e,fbf820e3286688cd] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.50], timeoutMillis=20000 2025-11-29 08:42:00.354 INFO 12717 --- [host-check-clockdiff4,f9f61431ecd5626e,8ab2688be8c330f8] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.52], timeoutMillis=20000 2025-11-29 08:42:00.355 INFO 12717 --- [host-check-clockdiff5,f9f61431ecd5626e,5c18f61ed25ec330] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.53], timeoutMillis=20000 2025-11-29 08:42:00.355 INFO 12717 --- [host-check-clockdiff6,f9f61431ecd5626e,eb93f7aa26b44fb8] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.54], timeoutMillis=20000 2025-11-29 08:42:00.355 INFO 12717 --- [host-check-clockdiff7,f9f61431ecd5626e,585868117d393394] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.59], timeoutMillis=20000 2025-11-29 08:42:00.355 INFO 12717 --- [host-check-clockdiff8,f9f61431ecd5626e,05a93cdec7004f10] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.60], timeoutMillis=20000 2025-11-29 08:42:00.355 INFO 12717 --- [host-check-clockdiff9,f9f61431ecd5626e,d64f1627ed342409] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.61], timeoutMillis=20000 2025-11-29 08:42:00.356 INFO 12717 --- [host-check-clockdiff10,f9f61431ecd5626e,584fbfefa9cd7b6c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.47], timeoutMillis=20000 2025-11-29 08:42:00.357 INFO 12717 --- [host-check-clockdiff11,f9f61431ecd5626e,49420eaaa30e346c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.48], timeoutMillis=20000 2025-11-29 08:42:00.357 INFO 12717 --- [host-check-agent-status8,f34d03ce53488cad,4e2711287d5f7e40] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:42:00.358 INFO 12717 --- [host-check-agent-status6,f34d03ce53488cad,f885af62d95080ef] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:42:00.358 INFO 12717 --- [host-check-agent-status7,f34d03ce53488cad,848b01ed731d26ef] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:42:00.358 INFO 12717 --- [host-check-agent-status2,f34d03ce53488cad,1251e06f40444bcf] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:42:00.363 INFO 12717 --- [host-check-clockdiff12,f9f61431ecd5626e,a51579d50d88ea79] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.49], timeoutMillis=20000 2025-11-29 08:42:00.364 INFO 12717 --- [host-check-agent-status10,f34d03ce53488cad,de7e39a9d1b5a613] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:42:00.365 INFO 12717 --- [host-check-agent-status5,f34d03ce53488cad,aab191bae4de797b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:42:00.365 INFO 12717 --- [host-check-agent-status6,f34d03ce53488cad,f885af62d95080ef] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:42:00.366 INFO 12717 --- [host-check-agent-status9,f34d03ce53488cad,c6a0d09e2fb7be58] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:42:00.366 INFO 12717 --- [host-check-agent-status4,f34d03ce53488cad,bd253c33dd971170] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:42:00.366 INFO 12717 --- [host-check-agent-status8,f34d03ce53488cad,4e2711287d5f7e40] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:42:00.367 INFO 12717 --- [host-check-agent-status7,f34d03ce53488cad,848b01ed731d26ef] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:42:00.367 INFO 12717 --- [host-check-agent-status2,f34d03ce53488cad,1251e06f40444bcf] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:42:00.367 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:42:00.367 INFO 12717 --- [host-check-agent-status3,f34d03ce53488cad,5cb52acbe33e498a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:42:00.367 INFO 12717 --- [host-check-agent-status1,f34d03ce53488cad,9ffae59262b6f892] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:42:00.368 INFO 12717 --- [host-check-agent-status10,f34d03ce53488cad,de7e39a9d1b5a613] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:42:00.368 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:42:00.368 INFO 12717 --- [host-check-agent-status11,f34d03ce53488cad,d18e250431584051] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:42:00.369 INFO 12717 --- [host-check-agent-status5,f34d03ce53488cad,aab191bae4de797b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:42:00.370 INFO 12717 --- [host-check-agent-status4,f34d03ce53488cad,bd253c33dd971170] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:42:00.372 INFO 12717 --- [host-check-agent-status9,f34d03ce53488cad,c6a0d09e2fb7be58] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:42:00.372 INFO 12717 --- [host-check-agent-status12,f34d03ce53488cad,db67e7f12af22a1e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:42:00.372 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:42:00.373 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:42:00.373 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:42:00.373 INFO 12717 --- [host-check-agent-status1,f34d03ce53488cad,9ffae59262b6f892] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:42:00.385 INFO 12717 --- [host-check-clockdiff1,f9f61431ecd5626e,49fcd7b1b0840134] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376920 0 0 , error= 2025-11-29 08:42:00.387 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:42:00.388 INFO 12717 --- [host-check-agent-status12,f34d03ce53488cad,db67e7f12af22a1e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:42:00.388 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:42:00.388 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:42:00.388 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:42:00.389 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:42:00.389 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:42:00.389 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:42:00.389 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:42:00.389 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:42:00.390 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:42:00.391 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:42:00.391 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:42:00.391 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:42:00.391 INFO 12717 --- [host-check-agent-status3,f34d03ce53488cad,5cb52acbe33e498a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:42:00.392 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:42:00.392 INFO 12717 --- [host-check-agent-status11,f34d03ce53488cad,d18e250431584051] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:42:00.392 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:42:00.393 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:42:00.395 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:42:00.396 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:42:00.397 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:42:00.513 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:00.513 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:00.514 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:42:00.514 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:42:00.515 INFO 12717 --- [host-check-agent-status7,f34d03ce53488cad,848b01ed731d26ef] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.515 INFO 12717 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:00.515 INFO 12717 --- [host-check-agent-status4,f34d03ce53488cad,bd253c33dd971170] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.516 INFO 12717 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [obconfig_url] 2025-11-29 08:42:00.516 INFO 12717 --- [host-check-agent-status2,f34d03ce53488cad,1251e06f40444bcf] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.516 INFO 12717 --- [host-check-agent-status9,f34d03ce53488cad,c6a0d09e2fb7be58] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.517 INFO 12717 --- [host-check-agent-status10,f34d03ce53488cad,de7e39a9d1b5a613] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.517 INFO 12717 --- [host-check-agent-status12,f34d03ce53488cad,db67e7f12af22a1e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.517 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/time, params:null 2025-11-29 08:42:00.517 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/time, params:null 2025-11-29 08:42:00.517 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/time, params:null 2025-11-29 08:42:00.517 INFO 12717 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.8 2025-11-29 08:42:00.518 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/time, params:null 2025-11-29 08:42:00.519 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/time, params:null 2025-11-29 08:42:00.519 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/time, params:null 2025-11-29 08:42:00.519 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.202.22.58, timestamp=2025-11-29T08:42:00.519416660+08:00 2025-11-29 08:42:00.520 INFO 12717 --- [host-check-agent-status4,f34d03ce53488cad,bd253c33dd971170] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.52 2025-11-29 08:42:00.520 INFO 12717 --- [host-check-agent-status4,f34d03ce53488cad,bd253c33dd971170] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.52 2025-11-29 08:42:00.520 INFO 12717 --- [host-check-agent-status4,f34d03ce53488cad,bd253c33dd971170] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.52 2025-11-29 08:42:00.520 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.202.22.54, timestamp=2025-11-29T08:42:00.517893266+08:00 2025-11-29 08:42:00.520 INFO 12717 --- [host-check-agent-status2,f34d03ce53488cad,1251e06f40444bcf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.50 2025-11-29 08:42:00.520 INFO 12717 --- [host-check-agent-status2,f34d03ce53488cad,1251e06f40444bcf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.50 2025-11-29 08:42:00.520 INFO 12717 --- [host-check-agent-status2,f34d03ce53488cad,1251e06f40444bcf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.50 2025-11-29 08:42:00.520 INFO 12717 --- [host-check-agent-status1,f34d03ce53488cad,9ffae59262b6f892] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.520 INFO 12717 --- [Check root service 2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:00.520 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/time, params:null 2025-11-29 08:42:00.521 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-11-29T08:42:00.519+08:00 2025-11-29 08:42:00.521 INFO 12717 --- [host-check-agent-status5,f34d03ce53488cad,aab191bae4de797b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.521 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.202.22.51, timestamp=2025-11-29T08:42:00.518589218+08:00 2025-11-29 08:42:00.521 INFO 12717 --- [host-check-agent-status8,f34d03ce53488cad,4e2711287d5f7e40] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.521 INFO 12717 --- [Check root service 2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [obconfig_url] 2025-11-29 08:42:00.521 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=13, ip=10.202.22.49, timestamp=2025-11-29T08:42:00.520033303+08:00 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-agent-status11,f34d03ce53488cad,d18e250431584051] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/time, params:null 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2025-11-29T08:42:00.520+08:00 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-11-29T08:42:00.521+08:00 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=13, lastAvailableTime=2025-11-29T08:42:00.521+08:00 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-agent-status3,f34d03ce53488cad,5cb52acbe33e498a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/time, params:null 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-agent-status10,f34d03ce53488cad,de7e39a9d1b5a613] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.47 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-agent-status10,f34d03ce53488cad,de7e39a9d1b5a613] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.47 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-agent-status10,f34d03ce53488cad,de7e39a9d1b5a613] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.47 2025-11-29 08:42:00.522 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=12, ip=10.202.22.48, timestamp=2025-11-29T08:42:00.520465241+08:00 2025-11-29 08:42:00.523 INFO 12717 --- [host-check-agent-status12,f34d03ce53488cad,db67e7f12af22a1e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.49 2025-11-29 08:42:00.523 INFO 12717 --- [host-check-agent-status12,f34d03ce53488cad,db67e7f12af22a1e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.49 2025-11-29 08:42:00.523 INFO 12717 --- [host-check-agent-status12,f34d03ce53488cad,db67e7f12af22a1e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.49 2025-11-29 08:42:00.523 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/time, params:null 2025-11-29 08:42:00.523 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.202.22.52, timestamp=2025-11-29T08:42:00.520875502+08:00 2025-11-29 08:42:00.523 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.202.22.53, timestamp=2025-11-29T08:42:00.522661342+08:00 2025-11-29 08:42:00.524 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-11-29T08:42:00.523+08:00 2025-11-29 08:42:00.524 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-11-29T08:42:00.519+08:00 and status AVAILABLE 2025-11-29 08:42:00.524 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=12, lastAvailableTime=2025-11-29T08:42:00.523+08:00 2025-11-29 08:42:00.524 INFO 12717 --- [host-check-agent-status1,f34d03ce53488cad,9ffae59262b6f892] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.58 2025-11-29 08:42:00.524 INFO 12717 --- [host-check-agent-status1,f34d03ce53488cad,9ffae59262b6f892] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.58 2025-11-29 08:42:00.524 INFO 12717 --- [host-check-agent-status1,f34d03ce53488cad,9ffae59262b6f892] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.58 2025-11-29 08:42:00.524 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-11-29T08:42:00.523+08:00 2025-11-29 08:42:00.525 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/time, params:null 2025-11-29 08:42:00.525 INFO 12717 --- [host-check-agent-status6,f34d03ce53488cad,f885af62d95080ef] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/agent/status, params:null 2025-11-29 08:42:00.525 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-11-29T08:42:00.520+08:00 and status AVAILABLE 2025-11-29 08:42:00.525 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-11-29T08:42:00.521+08:00 and status AVAILABLE 2025-11-29 08:42:00.526 INFO 12717 --- [host-check-agent-status11,f34d03ce53488cad,d18e250431584051] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.48 2025-11-29 08:42:00.526 INFO 12717 --- [host-check-agent-status11,f34d03ce53488cad,d18e250431584051] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.48 2025-11-29 08:42:00.526 INFO 12717 --- [host-check-agent-status11,f34d03ce53488cad,d18e250431584051] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.48 2025-11-29 08:42:00.526 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/time, params:null 2025-11-29 08:42:00.526 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.s.compute.HostAgentDaoManager : Updated agent 12 last available time 2025-11-29T08:42:00.521+08:00 and status AVAILABLE 2025-11-29 08:42:00.526 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-11-29T08:42:00.523+08:00 and status AVAILABLE 2025-11-29 08:42:00.526 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.202.22.50, timestamp=2025-11-29T08:42:00.525766146+08:00 2025-11-29 08:42:00.527 INFO 12717 --- [host-check-agent-status5,f34d03ce53488cad,aab191bae4de797b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.53 2025-11-29 08:42:00.527 INFO 12717 --- [host-check-agent-status5,f34d03ce53488cad,aab191bae4de797b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.53 2025-11-29 08:42:00.527 INFO 12717 --- [host-check-agent-status5,f34d03ce53488cad,aab191bae4de797b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.53 2025-11-29 08:42:00.527 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=11, ip=10.202.22.47, timestamp=2025-11-29T08:42:00.526542665+08:00 2025-11-29 08:42:00.527 INFO 12717 --- [host-check-agent-status3,f34d03ce53488cad,5cb52acbe33e498a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.51 2025-11-29 08:42:00.527 INFO 12717 --- [host-check-agent-status3,f34d03ce53488cad,5cb52acbe33e498a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.51 2025-11-29 08:42:00.527 INFO 12717 --- [host-check-agent-status3,f34d03ce53488cad,5cb52acbe33e498a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.51 2025-11-29 08:42:00.527 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-11-29T08:42:00.523+08:00 and status AVAILABLE 2025-11-29 08:42:00.527 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-11-29T08:42:00.527+08:00 2025-11-29 08:42:00.529 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.s.compute.HostAgentDaoManager : Updated agent 13 last available time 2025-11-29T08:42:00.523+08:00 and status AVAILABLE 2025-11-29 08:42:00.529 INFO 12717 --- [host-check-agent-status6,f34d03ce53488cad,f885af62d95080ef] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.54 2025-11-29 08:42:00.529 INFO 12717 --- [host-check-agent-status6,f34d03ce53488cad,f885af62d95080ef] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.54 2025-11-29 08:42:00.529 INFO 12717 --- [host-check-agent-status6,f34d03ce53488cad,f885af62d95080ef] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.54 2025-11-29 08:42:00.529 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=11, lastAvailableTime=2025-11-29T08:42:00.527+08:00 2025-11-29 08:42:00.531 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-77,7ca38b94f3a50b42,0c828fe2901ef329] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=ocpserver&ObClusterId=1743215593&version=2, client=10.202.22.58, traceId=7ca38b94f3a50b42, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getRootServiceInfo(String,Long,String,Long,boolean,Integer), args=ocpserver,1743215593,false,2,] 2025-11-29 08:42:00.531 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-77,7ca38b94f3a50b42,0c828fe2901ef329] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.58, traceId=7ca38b94f3a50b42, duration=1 ms] 2025-11-29 08:42:00.532 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2025-11-29T08:42:00.527+08:00 and status AVAILABLE 2025-11-29 08:42:00.533 INFO 12717 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:00.533 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.s.compute.HostAgentDaoManager : Updated agent 11 last available time 2025-11-29T08:42:00.527+08:00 and status AVAILABLE 2025-11-29 08:42:00.533 INFO 12717 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [rootservice_list] 2025-11-29 08:42:00.539 INFO 12717 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:00.542 INFO 12717 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.202.22.58, 2882] 2025-11-29 08:42:00.609 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=10, ip=10.202.22.61, timestamp=2025-11-29T08:42:00.606611042+08:00 2025-11-29 08:42:00.610 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=10, lastAvailableTime=2025-11-29T08:42:00.609+08:00 2025-11-29 08:42:00.613 INFO 12717 --- [host-check-agent-status9,f34d03ce53488cad,c6a0d09e2fb7be58] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.61 2025-11-29 08:42:00.613 INFO 12717 --- [host-check-agent-status9,f34d03ce53488cad,c6a0d09e2fb7be58] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.61 2025-11-29 08:42:00.613 INFO 12717 --- [host-check-agent-status9,f34d03ce53488cad,c6a0d09e2fb7be58] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.61 2025-11-29 08:42:00.615 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=9, ip=10.202.22.60, timestamp=2025-11-29T08:42:00.609070468+08:00 2025-11-29 08:42:00.616 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2025-11-29T08:42:00.609+08:00 and status AVAILABLE 2025-11-29 08:42:00.618 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=9, lastAvailableTime=2025-11-29T08:42:00.615+08:00 2025-11-29 08:42:00.620 INFO 12717 --- [host-check-agent-status8,f34d03ce53488cad,4e2711287d5f7e40] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.60 2025-11-29 08:42:00.620 INFO 12717 --- [host-check-agent-status8,f34d03ce53488cad,4e2711287d5f7e40] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.60 2025-11-29 08:42:00.620 INFO 12717 --- [host-check-agent-status8,f34d03ce53488cad,4e2711287d5f7e40] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.60 2025-11-29 08:42:00.623 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.s.compute.HostAgentDaoManager : Updated agent 10 last available time 2025-11-29T08:42:00.615+08:00 and status AVAILABLE 2025-11-29 08:42:00.626 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-11-29 08:42:00.626 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=12 2025-11-29 08:42:00.626 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-11-29 08:42:00.626 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-11-29 08:42:00.626 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2025-11-29 08:42:00.626 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=13 2025-11-29 08:42:00.626 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-11-29 08:42:00.627 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-11-29 08:42:00.627 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=11 2025-11-29 08:42:00.664 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:42:00.664 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:42:00.664 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:42:00.665 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:42:00.665 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:42:00.665 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:42:00.665 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:42:00.665 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:42:00.665 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:42:00.666 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:42:00.666 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:42:00.666 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:42:00.666 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:42:00.666 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:42:00.667 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:42:00.667 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:42:00.667 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:42:00.669 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:42:00.685 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.685 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.685 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.686 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.686 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.686 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.687 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.688 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.693 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.704 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=8, ip=10.202.22.59, timestamp=2025-11-29T08:42:00.703636969+08:00 2025-11-29 08:42:00.705 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=8, lastAvailableTime=2025-11-29T08:42:00.705+08:00 2025-11-29 08:42:00.706 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=10 2025-11-29 08:42:00.709 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=9 2025-11-29 08:42:00.709 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.s.compute.HostAgentDaoManager : Updated agent 8 last available time 2025-11-29T08:42:00.705+08:00 and status AVAILABLE 2025-11-29 08:42:00.710 INFO 12717 --- [host-check-agent-status7,f34d03ce53488cad,848b01ed731d26ef] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.59 2025-11-29 08:42:00.710 INFO 12717 --- [host-check-agent-status7,f34d03ce53488cad,848b01ed731d26ef] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.59 2025-11-29 08:42:00.710 INFO 12717 --- [host-check-agent-status7,f34d03ce53488cad,848b01ed731d26ef] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.59 2025-11-29 08:42:00.714 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=8 2025-11-29 08:42:00.715 INFO 12717 --- [Check status for all clusters2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.3.5.1 2025-11-29 08:42:00.776 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=16 2025-11-29 08:42:00.776 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-11-29 08:42:00.776 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=31 2025-11-29 08:42:00.776 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-11-29 08:42:00.776 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-11-29 08:42:00.776 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-11-29 08:42:00.776 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-11-29 08:42:00.776 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,4c6cdb032a10740f,ee7c74108f340a53] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=DB&ObClusterId=1743215594&version=2, client=10.202.22.58, traceId=4c6cdb032a10740f, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getRootServiceInfo(String,Long,String,Long,boolean,Integer), args=DB,1743215594,false,2,] 2025-11-29 08:42:00.776 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,4c6cdb032a10740f,ee7c74108f340a53] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.58, traceId=4c6cdb032a10740f, duration=0 ms] 2025-11-29 08:42:00.776 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=34 2025-11-29 08:42:00.787 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:42:00.788 INFO 12717 --- [Check root service 2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:00.789 INFO 12717 --- [Check root service 2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [rootservice_list] 2025-11-29 08:42:00.790 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:42:00.800 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:42:00.805 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:42:00.805 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:42:00.807 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:42:00.813 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-11-29 08:42:00.815 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=34 2025-11-29 08:42:00.815 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=16 2025-11-29 08:42:00.816 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-11-29 08:42:00.816 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=31 2025-11-29 08:42:00.818 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-11-29 08:42:00.818 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-11-29 08:42:00.819 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-11-29 08:42:00.837 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.841 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.841 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:00.881 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=37 2025-11-29 08:42:00.891 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=37 2025-11-29 08:42:00.984 INFO 12717 --- [ocp-schedules-24,963335a59d944f8e,9960e41cb5599206] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration auto replace status 2025-11-29 08:42:00.993 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:00.994 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:00.995 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:00.995 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:00.995 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:00.996 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:00.997 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:00.997 INFO 12717 --- [Check root service 2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:00.998 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:00.998 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:01.001 INFO 12717 --- [Check root service 2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.202.22.52, 2882] 2025-11-29 08:42:01.004 INFO 12717 --- [Check root service 2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:01.007 INFO 12717 --- [Check root service 2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.202.22.49, 2882] 2025-11-29 08:42:01.010 INFO 12717 --- [Check root service 2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:01.013 INFO 12717 --- [Check root service 2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.202.22.61, 2882] 2025-11-29 08:42:01.038 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=25 2025-11-29 08:42:01.045 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=17 2025-11-29 08:42:01.048 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=25 2025-11-29 08:42:01.049 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-11-29 08:42:01.049 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=28 2025-11-29 08:42:01.050 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=32 2025-11-29 08:42:01.050 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=35 2025-11-29 08:42:01.054 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=17 2025-11-29 08:42:01.056 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-11-29 08:42:01.058 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-11-29 08:42:01.058 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=28 2025-11-29 08:42:01.059 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=35 2025-11-29 08:42:01.059 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-11-29 08:42:01.060 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=32 2025-11-29 08:42:01.061 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-11-29 08:42:01.063 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-11-29 08:42:01.065 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-11-29 08:42:01.067 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:01.069 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-11-29 08:42:01.070 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-11-29 08:42:01.073 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-11-29 08:42:01.082 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.085 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.085 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.085 WARN 12717 --- [host-check-clockdiff10,f9f61431ecd5626e,584fbfefa9cd7b6c] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.47], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:01.085 ERROR 12717 --- [host-check-clockdiff10,f9f61431ecd5626e,584fbfefa9cd7b6c] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.47] 2025-11-29 08:42:01.085 WARN 12717 --- [host-check-clockdiff10,f9f61431ecd5626e,584fbfefa9cd7b6c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=11, ip=10.202.22.47, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.47 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:01.085 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:01.085 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.086 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.090 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.093 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.099 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.100 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=22 2025-11-29 08:42:01.112 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=22 2025-11-29 08:42:01.140 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=18 2025-11-29 08:42:01.141 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:01.143 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-11-29 08:42:01.147 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=33 2025-11-29 08:42:01.149 WARN 12717 --- [host-check-clockdiff3,f9f61431ecd5626e,231f786dd9227d05] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.51], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:01.149 ERROR 12717 --- [host-check-clockdiff3,f9f61431ecd5626e,231f786dd9227d05] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.51] 2025-11-29 08:42:01.149 WARN 12717 --- [host-check-clockdiff3,f9f61431ecd5626e,231f786dd9227d05] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.202.22.51, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.51 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:01.151 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=36 2025-11-29 08:42:01.151 INFO 12717 --- [host-check-host-status2,37a479e416bb1651,3b2175098f74550a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=18 2025-11-29 08:42:01.153 INFO 12717 --- [host-check-host-status3,37a479e416bb1651,84f87b4330a0dae8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-11-29 08:42:01.155 INFO 12717 --- [host-check-host-status10,37a479e416bb1651,acbdb1540a215748] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=33 2025-11-29 08:42:01.155 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-11-29 08:42:01.160 INFO 12717 --- [host-check-host-status12,37a479e416bb1651,449d65190320bc04] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=36 2025-11-29 08:42:01.164 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-11-29 08:42:01.164 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-11-29 08:42:01.164 INFO 12717 --- [host-check-host-status5,37a479e416bb1651,edb7d276188e222e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-11-29 08:42:01.168 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-11-29 08:42:01.172 INFO 12717 --- [host-check-host-status1,37a479e416bb1651,c04aa1af9abb71eb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-11-29 08:42:01.172 INFO 12717 --- [host-check-host-status6,37a479e416bb1651,d11bf37928b2898c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-11-29 08:42:01.176 INFO 12717 --- [host-check-host-status4,37a479e416bb1651,33b58e882e6b22c6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-11-29 08:42:01.189 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=23 2025-11-29 08:42:01.197 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=23 2025-11-29 08:42:01.265 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-100,78dfd9df29ff9be3,7b7d8c6c5590b688] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=78dfd9df29ff9be3, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:01.268 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-100,78dfd9df29ff9be3,7b7d8c6c5590b688] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=78dfd9df29ff9be3, duration=4 ms] 2025-11-29 08:42:01.271 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=38 2025-11-29 08:42:01.279 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=38 2025-11-29 08:42:01.335 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.337 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.341 WARN 12717 --- [host-check-clockdiff12,f9f61431ecd5626e,a51579d50d88ea79] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.49], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:01.341 ERROR 12717 --- [host-check-clockdiff12,f9f61431ecd5626e,a51579d50d88ea79] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.49] 2025-11-29 08:42:01.341 WARN 12717 --- [host-check-clockdiff12,f9f61431ecd5626e,a51579d50d88ea79] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=13, ip=10.202.22.49, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.49 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:01.356 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=26 2025-11-29 08:42:01.366 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=26 2025-11-29 08:42:01.369 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=29 2025-11-29 08:42:01.378 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=29 2025-11-29 08:42:01.382 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.393 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:01.433 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=27 2025-11-29 08:42:01.443 INFO 12717 --- [host-check-host-status9,37a479e416bb1651,c01d063928ea2763] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=27 2025-11-29 08:42:01.459 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=30 2025-11-29 08:42:01.468 INFO 12717 --- [host-check-host-status8,37a479e416bb1651,f5b5d08de272b1e9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=30 2025-11-29 08:42:01.471 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=24 2025-11-29 08:42:01.477 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:01.478 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:01.480 INFO 12717 --- [host-check-host-status7,37a479e416bb1651,e0d3faf50c1bc775] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=24 2025-11-29 08:42:01.531 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=39 2025-11-29 08:42:01.539 INFO 12717 --- [host-check-host-status11,37a479e416bb1651,7c0f82ccc15c30ae] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=39 2025-11-29 08:42:01.688 INFO 12717 --- [ocp-async-31,92806e7fb907daeb,3aa348910763f2b5] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:01.831 INFO 12717 --- [ocp-schedules-21,e8fff549d3dfe627,db8c6043131a7cc5] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:42:01.843 INFO 12717 --- [alarm-detect-7,e8fff549d3dfe627,dd28ab796b7fb274] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008306, data is empty. 2025-11-29 08:42:01.844 INFO 12717 --- [alarm-detect-19,e8fff549d3dfe627,55a937b6b2b5b6e9] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008306, data is empty. 2025-11-29 08:42:01.845 INFO 12717 --- [alarm-detect-15,e8fff549d3dfe627,a9916abce582025a] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008306, data is empty. 2025-11-29 08:42:01.849 INFO 12717 --- [alarm-detect-19,e8fff549d3dfe627,55a937b6b2b5b6e9] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008306, data is empty. 2025-11-29 08:42:01.850 WARN 12717 --- [scanner-11213807915-1,e8fff549d3dfe627,157eba12dd320f2b] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:42:01.853 INFO 12717 --- [alarm-detect-7,e8fff549d3dfe627,dd28ab796b7fb274] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008306, data is empty. 2025-11-29 08:42:01.855 INFO 12717 --- [alarm-detect-15,e8fff549d3dfe627,a9916abce582025a] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008306, data is empty. 2025-11-29 08:42:01.904 INFO 12717 --- [alarm-detect-5,e8fff549d3dfe627,cb2380d9fbb06160] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008306, data is empty. 2025-11-29 08:42:01.911 INFO 12717 --- [alarm-detect-5,e8fff549d3dfe627,cb2380d9fbb06160] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008306, data is empty. 2025-11-29 08:42:02.102 WARN 12717 --- [host-check-clockdiff6,f9f61431ecd5626e,eb93f7aa26b44fb8] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.54], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:02.102 ERROR 12717 --- [host-check-clockdiff6,f9f61431ecd5626e,eb93f7aa26b44fb8] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.54] 2025-11-29 08:42:02.102 WARN 12717 --- [host-check-clockdiff6,f9f61431ecd5626e,eb93f7aa26b44fb8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.202.22.54, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.54 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:02.132 WARN 12717 --- [host-check-clockdiff5,f9f61431ecd5626e,5c18f61ed25ec330] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.53], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:02.133 ERROR 12717 --- [host-check-clockdiff5,f9f61431ecd5626e,5c18f61ed25ec330] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.53] 2025-11-29 08:42:02.133 WARN 12717 --- [host-check-clockdiff5,f9f61431ecd5626e,5c18f61ed25ec330] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.202.22.53, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.53 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:02.266 WARN 12717 --- [host-check-clockdiff2,f9f61431ecd5626e,fbf820e3286688cd] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.50], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:02.267 ERROR 12717 --- [host-check-clockdiff2,f9f61431ecd5626e,fbf820e3286688cd] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.50] 2025-11-29 08:42:02.267 WARN 12717 --- [host-check-clockdiff2,f9f61431ecd5626e,fbf820e3286688cd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.202.22.50, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.50 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:02.289 WARN 12717 --- [host-check-clockdiff4,f9f61431ecd5626e,8ab2688be8c330f8] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.52], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:02.289 ERROR 12717 --- [host-check-clockdiff4,f9f61431ecd5626e,8ab2688be8c330f8] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.52] 2025-11-29 08:42:02.289 WARN 12717 --- [host-check-clockdiff4,f9f61431ecd5626e,8ab2688be8c330f8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.202.22.52, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.52 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:02.331 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-98,76feb7a136223423,83c671cf3c23cdf3] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=76feb7a136223423, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:02.331 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-98,76feb7a136223423,83c671cf3c23cdf3] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=76feb7a136223423, duration=1 ms] 2025-11-29 08:42:02.407 INFO 12717 --- [prometheus-minute-schedule-1,a4b2cfa928f851c4,fad7a5a7153a619c] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:02.413 INFO 12717 --- [prometheus-minute-schedule-1,a4b2cfa928f851c4,fad7a5a7153a619c] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:02.719 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,78d36f6dc0a41b20,78edf24fdefc94d8] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=78d36f6dc0a41b20, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:02.719 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,78d36f6dc0a41b20,78edf24fdefc94d8] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=78d36f6dc0a41b20, duration=1 ms] 2025-11-29 08:42:03.128 INFO 12717 --- [ocp-async-35,055470110d58ee90,1c8445c81210e89f] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:42:03.688 INFO 12717 --- [ocp-async-14,98938acf31b18a24,6d985ce688d34f3a] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:03.821 WARN 12717 --- [host-check-clockdiff7,f9f61431ecd5626e,585868117d393394] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.59], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:03.821 ERROR 12717 --- [host-check-clockdiff7,f9f61431ecd5626e,585868117d393394] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.59] 2025-11-29 08:42:03.821 WARN 12717 --- [host-check-clockdiff7,f9f61431ecd5626e,585868117d393394] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=8, ip=10.202.22.59, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.59 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:03.936 INFO 12717 --- [print-time-series-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Async compute queue max size = 0, current size = 0. 2025-11-29 08:42:03.982 INFO 12717 --- [ocp-schedules-3,f59e7d447795ed2e,af86ccab66181fc3] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog instances, size=0 2025-11-29 08:42:04.442 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,65366938aad57331,2b7c764388410c26] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=65366938aad57331, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:04.444 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,65366938aad57331,2b7c764388410c26] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=65366938aad57331, duration=3 ms] 2025-11-29 08:42:04.478 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:04.479 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:04.602 INFO 12717 --- [ocp-schedules-2,2d63bcb4c9ad3f7a,8ed25c40f23160dc] c.o.o.a.service.OcpAlarmSearchService : alarmTargetStat [] 2025-11-29 08:42:04.605 INFO 12717 --- [ocp-schedules-2,2d63bcb4c9ad3f7a,8ed25c40f23160dc] c.o.o.a.service.OcpAlarmSearchService : alarmTargetStat [] 2025-11-29 08:42:05.233 WARN 12717 --- [host-check-clockdiff11,f9f61431ecd5626e,49420eaaa30e346c] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.48], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:05.233 ERROR 12717 --- [host-check-clockdiff11,f9f61431ecd5626e,49420eaaa30e346c] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.48] 2025-11-29 08:42:05.233 WARN 12717 --- [host-check-clockdiff11,f9f61431ecd5626e,49420eaaa30e346c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=12, ip=10.202.22.48, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.48 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:05.407 INFO 12717 --- [prometheus-minute-schedule-1,6438a228b8f2552f,c81c513e0fcfc821] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:05.413 INFO 12717 --- [prometheus-minute-schedule-1,6438a228b8f2552f,c81c513e0fcfc821] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:05.688 INFO 12717 --- [ocp-async-33,791282cc855b29e8,9f4ee29f06227d5c] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:06.587 INFO 12717 --- [ocp-schedules-2,3870cd17e0568cb3,be6c910c62246ba0] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-11-29 08:42:06.753 INFO 12717 --- [ocp-schedules-2,3870cd17e0568cb3,be6c910c62246ba0] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:42:06.753 INFO 12717 --- [ocp-schedules-2,3870cd17e0568cb3,be6c910c62246ba0] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:42:07.116 INFO 12717 --- [ocp-schedules-3,cf96d1d587ecf902,716f0abb290a203e] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:42:07.119 INFO 12717 --- [ocp-schedules-3,cf96d1d587ecf902,716f0abb290a203e] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:42:07.479 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:07.480 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:07.688 INFO 12717 --- [ocp-async-21,45d079108070213a,d8a5d910700c5ab6] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:08.407 INFO 12717 --- [prometheus-minute-schedule-1,e96accd474a4e518,112279acfedc8c98] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:08.413 INFO 12717 --- [prometheus-minute-schedule-1,e96accd474a4e518,112279acfedc8c98] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:08.632 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,40a5ecc1f83c9d09,35a6089d147cae52] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=40a5ecc1f83c9d09, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:08.632 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,40a5ecc1f83c9d09,35a6089d147cae52] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=40a5ecc1f83c9d09, duration=1 ms] 2025-11-29 08:42:09.653 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:42:09.652418+08:00, lastProcessTime=2025-11-29T08:41:59.678702+08:00 2025-11-29 08:42:09.688 INFO 12717 --- [ocp-async-38,0ec704c696036614,3ffc72694e5518dd] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:09.828 INFO 12717 --- [host-check-clockdiff9,f9f61431ecd5626e,d64f1627ed342409] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376929 -1 -2 , error= 2025-11-29 08:42:10.017 INFO 12717 --- [ocp-schedules-15,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=3 2025-11-29 08:42:10.022 INFO 12717 --- [ocp-schedules-28,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=3 2025-11-29 08:42:10.042 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:42:09.652418+08:00 2025-11-29 08:42:10.051 INFO 12717 --- [Check obproxy1,ad407e73d2b413f2,14fc1a07e6652299] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:42:10.051 INFO 12717 --- [Check obproxy1,ad407e73d2b413f2,14fc1a07e6652299] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:10.052 INFO 12717 --- [Collect obproxy2,3702679cea1d7b42,cdaa17f0e4d97ce6] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:42:10.052 INFO 12717 --- [Collect obproxy2,3702679cea1d7b42,cdaa17f0e4d97ce6] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:10.052 INFO 12717 --- [Check obproxy2,0c6538831d07cac1,16eb55fcba36b291] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:42:10.052 INFO 12717 --- [Check obproxy2,0c6538831d07cac1,16eb55fcba36b291] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:10.053 INFO 12717 --- [Collect obproxy1,b27c8a0fc352410b,7c30098c73c960d8] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:42:10.053 INFO 12717 --- [Collect obproxy1,b27c8a0fc352410b,7c30098c73c960d8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:10.053 INFO 12717 --- [Check obproxy3,6ac9176cd84e3cee,13d5ab853b73fff0] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:42:10.053 INFO 12717 --- [Check obproxy3,6ac9176cd84e3cee,13d5ab853b73fff0] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:10.054 INFO 12717 --- [Collect obproxy3,96c1bf99040d439e,06f719f998af710d] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:42:10.054 INFO 12717 --- [Collect obproxy3,96c1bf99040d439e,06f719f998af710d] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:10.092 INFO 12717 --- [Check obproxy1,ad407e73d2b413f2,14fc1a07e6652299] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:42:10.094 INFO 12717 --- [Check obproxy3,6ac9176cd84e3cee,13d5ab853b73fff0] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:42:10.097 INFO 12717 --- [Check obproxy1,ad407e73d2b413f2,14fc1a07e6652299] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-11-29 08:42:10.097 INFO 12717 --- [Check obproxy1,ad407e73d2b413f2,14fc1a07e6652299] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 1, need to sync to meta: false 2025-11-29 08:42:10.097 INFO 12717 --- [Check obproxy3,6ac9176cd84e3cee,13d5ab853b73fff0] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2025-11-29 08:42:10.097 INFO 12717 --- [Check obproxy3,6ac9176cd84e3cee,13d5ab853b73fff0] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 3 proxy_id: 3, need to sync to meta: false 2025-11-29 08:42:10.099 INFO 12717 --- [Collect obproxy3,96c1bf99040d439e,06f719f998af710d] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:42:10.102 INFO 12717 --- [Collect obproxy2,3702679cea1d7b42,cdaa17f0e4d97ce6] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:42:10.102 INFO 12717 --- [Collect obproxy3,96c1bf99040d439e,06f719f998af710d] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:42:10.103 INFO 12717 --- [Check obproxy2,0c6538831d07cac1,16eb55fcba36b291] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:42:10.105 INFO 12717 --- [Collect obproxy3,96c1bf99040d439e,06f719f998af710d] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:42:10.106 INFO 12717 --- [Collect obproxy1,b27c8a0fc352410b,7c30098c73c960d8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:42:10.109 INFO 12717 --- [Check obproxy2,0c6538831d07cac1,16eb55fcba36b291] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-11-29 08:42:10.109 INFO 12717 --- [Check obproxy2,0c6538831d07cac1,16eb55fcba36b291] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 2, need to sync to meta: false 2025-11-29 08:42:10.110 INFO 12717 --- [Collect obproxy2,3702679cea1d7b42,cdaa17f0e4d97ce6] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:42:10.111 INFO 12717 --- [Collect obproxy1,b27c8a0fc352410b,7c30098c73c960d8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:42:10.115 INFO 12717 --- [Collect obproxy2,3702679cea1d7b42,cdaa17f0e4d97ce6] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:42:10.116 INFO 12717 --- [Collect obproxy1,b27c8a0fc352410b,7c30098c73c960d8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:42:10.145 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-89,efb90f26932789a7,3597d6fbdb6d84f2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=efb90f26932789a7, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:10.147 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-89,efb90f26932789a7,3597d6fbdb6d84f2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=efb90f26932789a7, duration=3 ms] 2025-11-29 08:42:10.163 INFO 12717 --- [ocp-schedules-17,c27e5934b5035394,325246b7c1c7d9bf] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:42:10.225 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-91,3b716deb693c15db,b25e53863cb3ad8e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=3b716deb693c15db, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:10.225 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-91,3b716deb693c15db,b25e53863cb3ad8e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=3b716deb693c15db, duration=0 ms] 2025-11-29 08:42:10.276 INFO 12717 --- [ocp-schedules-22,1377a7a917c32ed5,db218fa46084886a] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=2, successCount=2 2025-11-29 08:42:10.329 INFO 12717 --- [ocp-schedules-14,1b3447931d3892a7,7dcfce35e6119173] c.o.o.s.common.scheduled.PerfSchedules : Collect trans stat for alarm, time=2025-11-29T08:42:10.329+08:00, clusters=[1, 2] 2025-11-29 08:42:10.335 INFO 12717 --- [ocp-schedules-14,1b3447931d3892a7,7dcfce35e6119173] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster ocpserver, trans stat result is empty. 2025-11-29 08:42:10.341 INFO 12717 --- [ocp-schedules-14,1b3447931d3892a7,7dcfce35e6119173] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster DB, trans stat result is empty. 2025-11-29 08:42:10.480 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:10.481 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:10.527 INFO 12717 --- [ocp-schedules-25,ed9aad83ce02add8,a78eea2f192b08a9] .o.o.s.t.b.c.AbstractShardingClusterTask : Check cluster cgroup value. executed completed, allCount=2, successCount=2 2025-11-29 08:42:10.715 INFO 12717 --- [ocp-schedules-26,de9f021bfe81dbc0,40971b4ec3c02992] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:42:10.880 INFO 12717 --- [host-check-clockdiff8,f9f61431ecd5626e,05a93cdec7004f10] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376930 -5 -5 , error= 2025-11-29 08:42:10.950 INFO 12717 --- [async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 28, ruleSize = 28. 2025-11-29 08:42:11.015 INFO 12717 --- [ocp-schedules-11,8c7a43e4c5067e23,e18b447dc5e988fc] .o.o.s.t.b.c.AbstractShardingClusterTask : Check root service executed completed, allCount=2, successCount=2 2025-11-29 08:42:11.124 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-87,865f3de85519228d,d4dd9a9209899324] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=865f3de85519228d, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:11.125 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-87,865f3de85519228d,d4dd9a9209899324] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=865f3de85519228d, duration=1 ms] 2025-11-29 08:42:11.407 INFO 12717 --- [prometheus-minute-schedule-1,700935380fbdf070,b89d7b4ab1363783] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:11.414 INFO 12717 --- [prometheus-minute-schedule-1,700935380fbdf070,b89d7b4ab1363783] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:11.688 INFO 12717 --- [ocp-async-17,f4b2af6402606a7f,5d245d0fca536d55] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:11.856 INFO 12717 --- [metric-archive-second-21,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:11.857 INFO 12717 --- [metric-archive-second-22,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:11.859 INFO 12717 --- [metric-archive-second-10,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:11.860 INFO 12717 --- [metric-archive-second-32,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:11.861 INFO 12717 --- [metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:11.861 INFO 12717 --- [metric-archive-second-21,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=988, validCount=982 2025-11-29 08:42:11.863 INFO 12717 --- [metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:11.863 INFO 12717 --- [metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:11.863 INFO 12717 --- [metric-archive-second-22,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=988, validCount=984 2025-11-29 08:42:11.863 INFO 12717 --- [metric-archive-second-11,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=89, affectRows=89 2025-11-29 08:42:11.863 INFO 12717 --- [metric-archive-second-11,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=86, validCount=86 2025-11-29 08:42:11.864 INFO 12717 --- [metric-archive-second-10,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=998, validCount=997 2025-11-29 08:42:11.864 INFO 12717 --- [metric-archive-second-24,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:11.864 INFO 12717 --- [metric-archive-second-20,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:11.865 INFO 12717 --- [metric-archive-second-32,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=989, validCount=986 2025-11-29 08:42:11.865 INFO 12717 --- [metric-archive-second-15,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:11.866 INFO 12717 --- [metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=992, validCount=991 2025-11-29 08:42:11.868 INFO 12717 --- [metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=997, validCount=994 2025-11-29 08:42:11.868 INFO 12717 --- [metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=993, validCount=991 2025-11-29 08:42:11.870 INFO 12717 --- [metric-archive-second-24,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=990, validCount=990 2025-11-29 08:42:11.870 INFO 12717 --- [metric-archive-second-20,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=995, validCount=988 2025-11-29 08:42:11.870 INFO 12717 --- [metric-archive-second-15,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376860, nodeCount=1000, validCount=1000 2025-11-29 08:42:11.937 INFO 12717 --- [ocp-schedules-1,a27b4c16c7e1e38b,5c4b0a12c9d158ff] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:42:11.942 INFO 12717 --- [alarm-detect-4,a27b4c16c7e1e38b,0d05def5dcd1e141] c.o.o.alarm.core.detect.MetricDetector : alarmType host_agent_res_memory_over_threshold detect time range 1764376861 ~ 1764376921 matched, alarm-level: 3, evaluate result MultiExpressionEvaluateResult(metricSnapshots={host_agent_resident_memory_gb=[SeriesMetricValues(metric=Metric{name='host_agent_resident_memory_gb', labels={app=HOST, host=10.202.22.52, host_name=localhost.localdomain, ob_cluster_id=1743215594, ob_cluster_name=DB, obzone=, process=ocp_monagent, svr_ip=10.202.22.52}}, data=[ValueData(timestamp=1764376920, value=1.633)])]}, metricEvaluateResults={host_agent_resident_memory_gb > 1.5=[ExpressionEvaluateResult(matchValue=1.5, metric=host_agent_resident_memory_gb, match=true, value=1.633, values={1764376920=1.633}, valueCount=1, labels={app=HOST, host=10.202.22.52, host_name=localhost.localdomain, ob_cluster_id=1743215594, ob_cluster_name=DB, obzone=, process=ocp_monagent, svr_ip=10.202.22.52})]}, matched=true, isContainValidValue=null, message=null, level=3) 2025-11-29 08:42:11.950 WARN 12717 --- [scanner-11213810432-1,a27b4c16c7e1e38b,b36be5d0e567ddac] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:42:12.004 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,b737b764ac0102ce,1407bad72fdaf652] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=b737b764ac0102ce, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:12.005 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,b737b764ac0102ce,1407bad72fdaf652] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=b737b764ac0102ce, duration=2 ms] 2025-11-29 08:42:13.092 INFO 12717 --- [ocp-schedules-8,bb934530d3e4dddf,0b24a8e78eda32f9] c.o.o.s.c.scheduled.TenantSchedules : Check all tenant service name conflict 2025-11-29 08:42:13.092 INFO 12717 --- [ocp-schedules-8,bb934530d3e4dddf,0b24a8e78eda32f9] .t.CheckAllTenantServiceNameConflictTask : Check all tenant service name conflict. 2025-11-29 08:42:13.128 INFO 12717 --- [ocp-async-21,a48116b2e2a2aeaf,32d515cdd7db7a9f] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:42:13.481 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:13.482 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:13.688 INFO 12717 --- [ocp-async-31,81f861def1f27103,7841005a36f3b8fa] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:14.403 INFO 12717 --- [metric-archive-async-4,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1764376860, nodeCount=347, affectRows=347 2025-11-29 08:42:14.407 INFO 12717 --- [prometheus-minute-schedule-1,cfd63c64a6a7d3f3,9609b2a1bfbf5ba1] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:14.413 INFO 12717 --- [prometheus-minute-schedule-1,cfd63c64a6a7d3f3,9609b2a1bfbf5ba1] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:14.776 INFO 12717 --- [metric-archive-async-3,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1764376860, nodeCount=1000, affectRows=1000 2025-11-29 08:42:15.257 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,749264d3b5717d6c,be88f73e92360093] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=749264d3b5717d6c, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:15.261 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,749264d3b5717d6c,be88f73e92360093] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=749264d3b5717d6c, duration=5 ms] 2025-11-29 08:42:15.689 INFO 12717 --- [ocp-async-17,74f91e3a11fc817d,1f62395366fc0752] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:16.482 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:16.483 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:16.928 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,b26bd6c2fba0ceb2,1d5dc161573c30e3] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=b26bd6c2fba0ceb2, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:16.928 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,b26bd6c2fba0ceb2,1d5dc161573c30e3] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=b26bd6c2fba0ceb2, duration=1 ms] 2025-11-29 08:42:17.126 INFO 12717 --- [ocp-schedules-22,71a3e979323e7e18,9949737035ace19b] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:42:17.129 INFO 12717 --- [ocp-schedules-22,71a3e979323e7e18,9949737035ace19b] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:42:17.407 INFO 12717 --- [prometheus-minute-schedule-1,bc0f81222e2ef9bf,e377da51717a0a67] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:17.414 INFO 12717 --- [prometheus-minute-schedule-1,bc0f81222e2ef9bf,e377da51717a0a67] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:17.689 INFO 12717 --- [ocp-async-34,4a641ec823f57ca9,24c04eee7aa9abc6] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:17.734 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,20a7a6b0a23cde27,b94fa0a9444f325e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=20a7a6b0a23cde27, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:17.734 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,20a7a6b0a23cde27,b94fa0a9444f325e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=20a7a6b0a23cde27, duration=1 ms] 2025-11-29 08:42:17.943 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule start 2025-11-29 08:42:17.984 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=1, name=ocpserver, obClusterId=1743215593, status=RUNNING} 2025-11-29 08:42:18.005 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING} 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-8) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-5) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-2) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-4) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-1) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-11) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-10) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-7) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-9) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (ocpserver:1743215593-1) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-6) 2025-11-29 08:42:18.021 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-3) 2025-11-29 08:42:18.028 INFO 12717 --- [sql-rollup-worker-26,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=7, level=0->1, c=0) - not enough cached interval (source=2025-11-29T08:39:56, target=2025-11-29T08:37:56, granule=120000ms, delay=120000ms) 2025-11-29 08:42:18.029 INFO 12717 --- [sql-rollup-worker-26,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=7, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:37:56, target=2025-11-29T08:28:25, granule=600000ms, delay=571000ms) 2025-11-29 08:42:18.029 INFO 12717 --- [sql-rollup-worker-32,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=6, level=0->1, c=0) - not enough cached interval (source=2025-11-29T08:39:46, target=2025-11-29T08:38:16, granule=120000ms, delay=90000ms) 2025-11-29 08:42:18.029 INFO 12717 --- [sql-rollup-worker-32,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=6, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:38:16, target=2025-11-29T08:28:16, granule=600000ms, delay=600000ms) 2025-11-29 08:42:18.029 INFO 12717 --- [sql-rollup-worker-17,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=4, level=0->1, c=0) - not enough cached interval (source=2025-11-29T08:39:57, target=2025-11-29T08:37:57, granule=120000ms, delay=120000ms) 2025-11-29 08:42:18.029 INFO 12717 --- [sql-rollup-worker-17,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=4, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:37:57, target=2025-11-29T08:30:57, granule=600000ms, delay=420000ms) 2025-11-29 08:42:18.029 INFO 12717 --- [sql-rollup-worker-35,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=2, level=0->1, c=0) - not enough cached interval (source=2025-11-29T08:40:06, target=2025-11-29T08:38:06, granule=120000ms, delay=120000ms) 2025-11-29 08:42:18.029 INFO 12717 --- [sql-rollup-worker-35,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=2, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:38:06, target=2025-11-29T08:24:37, granule=600000ms, delay=809000ms) 2025-11-29 08:42:18.029 INFO 12717 --- [sql-rollup-worker-21,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-24,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-33,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-19,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-34,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-30,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-31,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-29,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-21,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-24,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-34,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:42:18.030 INFO 12717 --- [sql-rollup-worker-33,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:42:18.031 INFO 12717 --- [sql-rollup-worker-19,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:42:18.031 INFO 12717 --- [sql-rollup-worker-30,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:42:18.031 INFO 12717 --- [sql-rollup-worker-29,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:42:18.031 INFO 12717 --- [sql-rollup-worker-31,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:42:18.033 INFO 12717 --- [sql-rollup-worker-33,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.033 INFO 12717 --- [sql-rollup-worker-24,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.033 INFO 12717 --- [sql-rollup-worker-21,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.033 INFO 12717 --- [sql-rollup-worker-34,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.033 INFO 12717 --- [sql-rollup-worker-30,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.033 INFO 12717 --- [sql-rollup-worker-29,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.034 INFO 12717 --- [sql-rollup-worker-19,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.034 INFO 12717 --- [sql-rollup-worker-24,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:18.034 INFO 12717 --- [sql-rollup-worker-33,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:18.034 INFO 12717 --- [sql-rollup-worker-21,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:18.034 INFO 12717 --- [sql-rollup-worker-31,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:18.034 INFO 12717 --- [sql-rollup-worker-29,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:18.034 INFO 12717 --- [sql-rollup-worker-34,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:18.034 INFO 12717 --- [sql-rollup-worker-19,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:18.034 INFO 12717 --- [sql-rollup-worker-30,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:18.034 INFO 12717 --- [sql-rollup-worker-31,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:18.035 INFO 12717 --- [sql-rollup-worker-19,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup newly (cluster=ocpserver:1743215593, server=1, level=0->1, c=0) - not enough cached interval (source=2025-03-29T10:42:31, earliest=2025-11-28T08:42:18, granule=120000ms, delay=-21074387000ms) 2025-11-29 08:42:18.035 INFO 12717 --- [sql-rollup-worker-19,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup greedily (cluster=ocpserver:1743215593, server=1, level=1->2, c=0) - source level not data, skipped 2025-11-29 08:42:18.051 ERROR 12717 --- [sql-rollup-worker-34,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup server-side (cluster=DB:1743215594, server=3, level=0->1, c=0) - source level expect data, actual none (from=2025-11-29T08:37:03) 2025-11-29 08:42:18.073 INFO 12717 --- [sql-rollup-worker-34,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=3, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:40:02, target=2025-11-29T08:39:03, granule=120000ms, delay=59000ms) 2025-11-29 08:42:18.073 INFO 12717 --- [sql-rollup-worker-34,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=3, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:39:03, target=2025-11-29T08:30:33, granule=600000ms, delay=510000ms) 2025-11-29 08:42:18.103 INFO 12717 --- [sql-rollup-worker-29,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=5, level=0->1, c=0) - source=2025-11-29T08:39:55, start=2025-11-29T08:35:25, end=2025-11-29T08:37:55, updates=70, granule=120000ms, delay=270000ms 2025-11-29 08:42:18.109 INFO 12717 --- [sql-rollup-worker-21,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=1, level=0->1, c=0) - source=2025-11-29T08:40:10, start=2025-11-29T08:37:40, end=2025-11-29T08:40:10, updates=69, granule=120000ms, delay=150000ms 2025-11-29 08:42:18.111 INFO 12717 --- [sql-rollup-worker-33,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=11, level=0->1, c=0) - source=2025-11-29T08:39:55, start=2025-11-29T08:36:55, end=2025-11-29T08:39:25, updates=73, granule=120000ms, delay=180000ms 2025-11-29 08:42:18.118 INFO 12717 --- [sql-rollup-worker-29,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=5, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:39:55, target=2025-11-29T08:37:55, granule=120000ms, delay=120000ms) 2025-11-29 08:42:18.118 INFO 12717 --- [sql-rollup-worker-29,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=5, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:37:55, target=2025-11-29T08:28:54, granule=600000ms, delay=541000ms) 2025-11-29 08:42:18.121 INFO 12717 --- [sql-rollup-worker-21,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=1, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:40:10, target=2025-11-29T08:40:09, granule=120000ms, delay=1000ms) 2025-11-29 08:42:18.121 INFO 12717 --- [sql-rollup-worker-30,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=8, level=0->1, c=0) - source=2025-11-29T08:39:47, start=2025-11-29T08:37:17, end=2025-11-29T08:39:47, updates=97, granule=120000ms, delay=150000ms 2025-11-29 08:42:18.121 INFO 12717 --- [sql-rollup-worker-21,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=1, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:40:09, target=2025-11-29T08:29:09, granule=600000ms, delay=660000ms) 2025-11-29 08:42:18.121 INFO 12717 --- [sql-rollup-worker-33,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=11, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:39:55, target=2025-11-29T08:39:25, granule=120000ms, delay=30000ms) 2025-11-29 08:42:18.121 INFO 12717 --- [sql-rollup-worker-33,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=11, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:39:25, target=2025-11-29T08:27:25, granule=600000ms, delay=720000ms) 2025-11-29 08:42:18.123 INFO 12717 --- [sql-rollup-worker-24,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=9, level=0->1, c=0) - source=2025-11-29T08:40:11, start=2025-11-29T08:37:11, end=2025-11-29T08:39:41, updates=81, granule=120000ms, delay=180000ms 2025-11-29 08:42:18.127 INFO 12717 --- [sql-rollup-worker-30,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=8, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:39:47, target=2025-11-29T08:39:47, granule=120000ms, delay=0ms) 2025-11-29 08:42:18.127 INFO 12717 --- [sql-rollup-worker-30,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=8, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:39:47, target=2025-11-29T08:29:17, granule=600000ms, delay=630000ms) 2025-11-29 08:42:18.129 INFO 12717 --- [sql-rollup-worker-24,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=9, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:40:11, target=2025-11-29T08:39:41, granule=120000ms, delay=30000ms) 2025-11-29 08:42:18.129 INFO 12717 --- [sql-rollup-worker-24,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=9, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:39:41, target=2025-11-29T08:29:10, granule=600000ms, delay=631000ms) 2025-11-29 08:42:18.133 INFO 12717 --- [sql-rollup-worker-31,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=10, level=0->1, c=0) - source=2025-11-29T08:40:09, start=2025-11-29T08:37:39, end=2025-11-29T08:40:09, updates=106, granule=120000ms, delay=150000ms 2025-11-29 08:42:18.137 INFO 12717 --- [sql-rollup-worker-31,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=10, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:40:09, target=2025-11-29T08:40:09, granule=120000ms, delay=0ms) 2025-11-29 08:42:18.210 INFO 12717 --- [sql-rollup-worker-31,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=10, level=1->2, c=0) - source=2025-11-29T08:40:09, start=2025-11-29T08:26:09, end=2025-11-29T08:36:09, updates=151, granule=600000ms, delay=840000ms 2025-11-29 08:42:18.214 INFO 12717 --- [sql-rollup-worker-31,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=10, level=1->2, c=1) - not enough cached interval (source=2025-11-29T08:40:09, target=2025-11-29T08:36:09, granule=600000ms, delay=240000ms) 2025-11-29 08:42:18.214 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule end, elapsed=271ms, period=120000ms 2025-11-29 08:42:18.907 INFO 12717 --- [ocp-schedules-8,39b2169e6c0078ee,d5f050c24449ea96] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-11-29 08:42:19.280 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.o.s.common.scheduled.CacheSchedules : Refresh cache bean: obQueryCache 2025-11-29 08:42:19.280 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.ocp.core.cache.helper.ObQueryCache : Init ob cache. 2025-11-29 08:42:19.302 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.ocp.core.cache.helper.ObQueryCache : Finished init ob cache. 2025-11-29 08:42:19.302 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.o.s.common.scheduled.CacheSchedules : Refresh cache bean: obProxyQueryCache 2025-11-29 08:42:19.302 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.o.c.cache.helper.ObProxyQueryCache : Init obproxy cache. 2025-11-29 08:42:19.329 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.o.c.cache.helper.ObProxyQueryCache : Finished init obproxy cache. 2025-11-29 08:42:19.329 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.o.s.common.scheduled.CacheSchedules : Refresh cache bean: hostQueryCache 2025-11-29 08:42:19.329 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.o.core.cache.helper.HostQueryCache : Init host cache. 2025-11-29 08:42:19.334 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.o.core.cache.helper.HostQueryCache : Finished init host cache. 2025-11-29 08:42:19.334 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.o.s.common.scheduled.CacheSchedules : Refresh cache bean: binlogQueryCache 2025-11-29 08:42:19.334 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.o.c.cache.helper.BinlogQueryCache : Init binlog cache. 2025-11-29 08:42:19.345 INFO 12717 --- [ocp-schedules-17,30c6c1cbf6b91550,243b4ce53a6c7181] c.o.o.c.cache.helper.BinlogQueryCache : Finish init binlog cache. 2025-11-29 08:42:19.483 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:19.484 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:19.676 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:42:19.675702+08:00, lastProcessTime=2025-11-29T08:42:09.652418+08:00 2025-11-29 08:42:19.689 INFO 12717 --- [ocp-async-28,ff09141100edfdbe,370c31060ddf89b6] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:19.901 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:42:19.675702+08:00 2025-11-29 08:42:19.917 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,db09ffb6f2c4a8df,9d41dc5771ea758f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=db09ffb6f2c4a8df, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:19.917 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,db09ffb6f2c4a8df,9d41dc5771ea758f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=db09ffb6f2c4a8df, duration=1 ms] 2025-11-29 08:42:20.168 INFO 12717 --- [ocp-schedules-5,d7b8de0695c4cc5e,ecd6a9b79e0e6dff] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:42:20.407 INFO 12717 --- [prometheus-minute-schedule-1,dd6e0a01e697a642,46ffd1eaf17815b2] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:20.413 INFO 12717 --- [prometheus-minute-schedule-1,dd6e0a01e697a642,46ffd1eaf17815b2] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:21.063 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,1b54b5db553803d1,f828bad0119ea761] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=1b54b5db553803d1, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:21.066 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,1b54b5db553803d1,f828bad0119ea761] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=1b54b5db553803d1, duration=4 ms] 2025-11-29 08:42:21.689 INFO 12717 --- [ocp-async-23,de3e2d65eeccf1d0,c92358902f6e4dca] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:22.052 INFO 12717 --- [ocp-schedules-6,458bd0036605368f,747fecc92674f104] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:42:22.058 INFO 12717 --- [alarm-detect-1,458bd0036605368f,2342d77b7b19c574] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008306, data is empty. 2025-11-29 08:42:22.060 INFO 12717 --- [alarm-detect-17,458bd0036605368f,0de84b4c05784270] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008306, data is empty. 2025-11-29 08:42:22.062 INFO 12717 --- [alarm-detect-11,458bd0036605368f,db6890c6c036a329] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008306, data is empty. 2025-11-29 08:42:22.063 INFO 12717 --- [alarm-detect-1,458bd0036605368f,2342d77b7b19c574] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008306, data is empty. 2025-11-29 08:42:22.066 INFO 12717 --- [alarm-detect-17,458bd0036605368f,0de84b4c05784270] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008306, data is empty. 2025-11-29 08:42:22.076 INFO 12717 --- [alarm-detect-11,458bd0036605368f,db6890c6c036a329] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008306, data is empty. 2025-11-29 08:42:22.086 WARN 12717 --- [scanner-11213810693-1,458bd0036605368f,b92059cdbe01094b] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:42:22.139 INFO 12717 --- [alarm-detect-5,458bd0036605368f,d8ad3c5392ef1a40] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008306, data is empty. 2025-11-29 08:42:22.143 INFO 12717 --- [alarm-detect-5,458bd0036605368f,d8ad3c5392ef1a40] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008306, data is empty. 2025-11-29 08:42:22.484 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:22.485 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:22.608 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,bd8f7007211456a3,8b43252385f5078e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=bd8f7007211456a3, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:22.610 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,bd8f7007211456a3,8b43252385f5078e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=bd8f7007211456a3, duration=2 ms] 2025-11-29 08:42:23.128 INFO 12717 --- [ocp-async-33,2268e99ba4b2e3df,35116e74e06cebb7] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:42:23.407 INFO 12717 --- [prometheus-minute-schedule-1,1cd724889efbccdd,4717964cea167518] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:23.413 INFO 12717 --- [prometheus-minute-schedule-1,1cd724889efbccdd,4717964cea167518] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:23.689 INFO 12717 --- [ocp-async-25,5842eb26d41b30c7,60c1736d41a4929e] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:25.143 INFO 12717 --- [ocp-schedules-28,cd5367c0085418e2,c69b048a74428d04] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info 2025-11-29 08:42:25.154 INFO 12717 --- [ocp-schedules-28,cd5367c0085418e2,c69b048a74428d04] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Invalidate clusters [] cache info success. 2025-11-29 08:42:25.154 INFO 12717 --- [ocp-schedules-28,cd5367c0085418e2,c69b048a74428d04] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info success, cached size: 2 2025-11-29 08:42:25.225 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,6161f14daa666956,261b74d5fb034c12] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=6161f14daa666956, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:25.225 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,6161f14daa666956,261b74d5fb034c12] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=6161f14daa666956, duration=1 ms] 2025-11-29 08:42:25.246 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,91a68e774a15db66,4bf9d71ad9737d0c] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=91a68e774a15db66, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:25.246 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,91a68e774a15db66,4bf9d71ad9737d0c] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=91a68e774a15db66, duration=1 ms] 2025-11-29 08:42:25.246 INFO 12717 --- [ocp-schedules-22,8c8f597f9e7271c3,bb309bbc95fd9dc2] c.o.o.s.common.scheduled.PerfSchedules : Oas sql curing, clusters=[] 2025-11-29 08:42:25.361 INFO 12717 --- [ocp-schedules-12,6a7f1a83ea7d2cdc,97f399254c771108] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:42:25.361 INFO 12717 --- [ocp-schedules-12,f361c6ff313d51b1,18907943ad2490e2] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-11-29 08:42:25.363 INFO 12717 --- [ocp-schedules-12,f361c6ff313d51b1,18907943ad2490e2] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:42:25.405 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:25.405 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:42:25.411 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:25.411 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:42:25.441 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:25.442 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:25.442 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:42:25.478 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:25.479 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:25.479 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:42:25.481 WARN 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=OCP 2025-11-29 08:42:25.481 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:25.484 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:42:25.485 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:25.485 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:42:25.487 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:25.496 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.496 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:25.602 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:25.604 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:25.604 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:42:25.637 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.648 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:25.649 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:25.650 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:42:25.651 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster ocpserver 2025-11-29 08:42:25.681 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2025-11-29 08:42:25.686 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:42:25.690 INFO 12717 --- [ocp-async-8,75aeda9f3ec5ccbc,2a83e0607ca8d7e1] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:25.691 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:42:25.766 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:25.767 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:25.767 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:42:25.772 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:25.775 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:42:25.779 INFO 12717 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:42:25.783 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:42:25.798 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.799 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:25.856 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.856 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:12+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:25.875 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.876 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:25.882 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=8073, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:42:25.891 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:25.891 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:25.892 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:42:25.893 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.893 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:42:25.901 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=ocpserver, obClusterId=1743215593 2025-11-29 08:42:25.901 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=ocpserver, obClusterId=1743215593, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"ocpserver","ObClusterId":1743215593,"ObRegion":"ocpserver","ObRegionId":1743215593,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.58:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376673675000}, createTime=2025-03-29T10:43:06+08:00, creator=admin, obVersion=4.2.1.8, packageOperatingSystem=null, loadType=null, tenantCount=3) 2025-11-29 08:42:25.906 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.906 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:25.921 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.921 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:25.937 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.937 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:25.955 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.955 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:30+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:25.971 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.971 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:25.989 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:25.989 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:42:26.003 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:26.003 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:26.224 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:26.224 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:42+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:26.240 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:26.240 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:26.255 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:26.268 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:26.269 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:26.270 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:42:26.357 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster DB 2025-11-29 08:42:26.362 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 2025-11-29 08:42:26.367 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:42:26.368 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:42:26.385 INFO 12717 --- [Sync all clusters2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:42:26.407 INFO 12717 --- [prometheus-minute-schedule-1,47781bbe85963d3a,8126b6d3f053274a] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:26.415 INFO 12717 --- [prometheus-minute-schedule-1,47781bbe85963d3a,8126b6d3f053274a] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:26.540 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 got ObProcessInfoResult(pid=2447, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:42:26.554 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:26.555 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:26.556 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:42:26.570 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=DB, obClusterId=1743215594 2025-11-29 08:42:26.570 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=DB, obClusterId=1743215594, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=CREATED, syncStatus=VALID, rootserverJson={"ObCluster":"DB","ObClusterId":1743215594,"ObRegion":"DB","ObRegionId":1743215594,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.52:2882","role":"LEADER","sql_port":2881},{"address":"10.202.22.49:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.202.22.61:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376629943000}, createTime=2025-03-29T11:15:04+08:00, creator=admin, obVersion=4.3.5.1, packageOperatingSystem=el8, loadType=HTAP, tenantCount=7) 2025-11-29 08:42:26.756 INFO 12717 --- [ocp-schedules-1,ac7ef27e5177dfbc,e9ee2e287de91a38] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-11-29 08:42:26.959 INFO 12717 --- [ocp-schedules-1,ac7ef27e5177dfbc,e9ee2e287de91a38] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:42:26.959 INFO 12717 --- [ocp-schedules-1,ac7ef27e5177dfbc,e9ee2e287de91a38] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:42:27.325 INFO 12717 --- [ocp-schedules-24,ed14f8da7fa582ea,8512b82f07d979e0] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:42:27.328 INFO 12717 --- [ocp-schedules-24,ed14f8da7fa582ea,8512b82f07d979e0] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:42:27.643 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,16c4d3a0ccbf6004,b85547eced6e6df2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=16c4d3a0ccbf6004, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:27.649 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,16c4d3a0ccbf6004,b85547eced6e6df2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=16c4d3a0ccbf6004, duration=6 ms] 2025-11-29 08:42:27.690 INFO 12717 --- [ocp-async-8,e2726ba880024a7c,26b9f1034870ae42] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:28.487 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:28.488 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:28.710 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,5bb0a261bee592e1,4a3c30ab33a26826] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=5bb0a261bee592e1, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:28.710 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,5bb0a261bee592e1,4a3c30ab33a26826] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=5bb0a261bee592e1, duration=1 ms] 2025-11-29 08:42:29.407 INFO 12717 --- [prometheus-minute-schedule-1,b8bfe3cb6b81e2cb,c52b840521f1ea82] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:29.414 INFO 12717 --- [prometheus-minute-schedule-1,b8bfe3cb6b81e2cb,c52b840521f1ea82] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:29.683 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:42:29.682767+08:00, lastProcessTime=2025-11-29T08:42:19.675702+08:00 2025-11-29 08:42:29.690 INFO 12717 --- [ocp-async-32,5cf5dd97b1e7b4b5,40ff568918482431] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:29.964 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:42:29.682767+08:00 2025-11-29 08:42:30.174 INFO 12717 --- [ocp-schedules-5,3709dc37e29d3df1,cacfe9302bbad0e0] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:42:30.271 INFO 12717 --- [ocp-schedules-1,6c5d0c18e7b4fc36,60a1ed5173814584] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-11-29 08:42:30.271 INFO 12717 --- [ocp-schedules-27,761dbf9ef1147d3e,62cb0d2084b64260] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-11-29 08:42:30.271 INFO 12717 --- [ocp-schedules-16,7c5716471e615789,8c54425f9fa8243f] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-11-29 08:42:30.273 INFO 12717 --- [ocp-schedules-27,761dbf9ef1147d3e,62cb0d2084b64260] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:42:30.274 INFO 12717 --- [ocp-schedules-16,7c5716471e615789,8c54425f9fa8243f] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:42:30.295 INFO 12717 --- [ocp-schedules-3,39099e0f1d14d53d,62fa6d7621898570] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-11-29 08:42:30.299 INFO 12717 --- [ocp-schedules-22,2d6619c1f8c43c40,6d9d09edb1e4cefd] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:42:30.299 INFO 12717 --- [host-check-clockdiff1,2d6619c1f8c43c40,8cfd7bdc5fe0f714] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.58], timeoutMillis=20000 2025-11-29 08:42:30.299 INFO 12717 --- [host-check-clockdiff2,2d6619c1f8c43c40,dd7936ff04371204] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.50], timeoutMillis=20000 2025-11-29 08:42:30.300 INFO 12717 --- [host-check-clockdiff3,2d6619c1f8c43c40,e310f235d44e7ba0] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.51], timeoutMillis=20000 2025-11-29 08:42:30.300 INFO 12717 --- [host-check-clockdiff4,2d6619c1f8c43c40,251b49cdb8b28bed] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.52], timeoutMillis=20000 2025-11-29 08:42:30.300 INFO 12717 --- [host-check-clockdiff5,2d6619c1f8c43c40,95b5af83df7db142] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.53], timeoutMillis=20000 2025-11-29 08:42:30.300 INFO 12717 --- [host-check-clockdiff6,2d6619c1f8c43c40,23ce851b5783ea1b] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.54], timeoutMillis=20000 2025-11-29 08:42:30.300 INFO 12717 --- [host-check-clockdiff7,2d6619c1f8c43c40,138532199869dbc9] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.59], timeoutMillis=20000 2025-11-29 08:42:30.301 INFO 12717 --- [host-check-clockdiff8,2d6619c1f8c43c40,ef8fbb32494a5ad3] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.60], timeoutMillis=20000 2025-11-29 08:42:30.301 INFO 12717 --- [host-check-clockdiff9,2d6619c1f8c43c40,7a2af28fb30f9e18] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.61], timeoutMillis=20000 2025-11-29 08:42:30.301 INFO 12717 --- [host-check-clockdiff10,2d6619c1f8c43c40,752fb5985b0a3025] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.47], timeoutMillis=20000 2025-11-29 08:42:30.301 INFO 12717 --- [host-check-clockdiff11,2d6619c1f8c43c40,45ac1eb191929386] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.48], timeoutMillis=20000 2025-11-29 08:42:30.301 INFO 12717 --- [host-check-clockdiff12,2d6619c1f8c43c40,b17e06bb099d18bc] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.49], timeoutMillis=20000 2025-11-29 08:42:30.307 INFO 12717 --- [ocp-schedules-25,f0b6d6db80f198aa,c4b4c7d3c295bde2] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:42:30.308 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:42:30.308 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:42:30.308 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:42:30.308 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:42:30.308 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:42:30.309 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:42:30.309 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:42:30.309 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:42:30.309 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:42:30.309 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:42:30.310 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:42:30.310 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:42:30.346 INFO 12717 --- [ocp-schedules-19,ae80993d3579a031,248e626180a85aca] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:42:30.346 INFO 12717 --- [host-check-agent-status1,ae80993d3579a031,97681c07f3448214] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:42:30.346 INFO 12717 --- [host-check-agent-status2,ae80993d3579a031,d748c911c6cb2f7b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:42:30.347 INFO 12717 --- [host-check-agent-status4,ae80993d3579a031,3a737440997f9a62] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:42:30.347 INFO 12717 --- [host-check-agent-status5,ae80993d3579a031,dba2b9104e9a5e0e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:42:30.347 INFO 12717 --- [host-check-agent-status3,ae80993d3579a031,75fb3d154728613c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:42:30.347 INFO 12717 --- [host-check-agent-status6,ae80993d3579a031,e3dd83b3e0a3fbd3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:42:30.348 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:42:30.348 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:42:30.348 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:42:30.348 INFO 12717 --- [host-check-agent-status7,ae80993d3579a031,37b0db692df7e393] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:42:30.348 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:42:30.349 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:42:30.349 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:42:30.349 INFO 12717 --- [host-check-agent-status10,ae80993d3579a031,ab26817a786fe25e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:42:30.349 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:42:30.349 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:42:30.349 INFO 12717 --- [host-check-agent-status9,ae80993d3579a031,75fa515c9adef66e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:42:30.350 INFO 12717 --- [host-check-agent-status11,ae80993d3579a031,fe56842de3355579] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:42:30.350 INFO 12717 --- [host-check-agent-status12,ae80993d3579a031,90468e4b30628f3f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:42:30.350 INFO 12717 --- [host-check-agent-status8,ae80993d3579a031,dc4b334f1e7dc285] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:42:30.354 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:42:30.373 INFO 12717 --- [host-check-clockdiff1,2d6619c1f8c43c40,8cfd7bdc5fe0f714] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376950 0 0 , error= 2025-11-29 08:42:30.373 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:42:30.373 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:42:30.374 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:42:30.374 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:42:30.374 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:42:30.375 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:42:30.376 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:42:30.376 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:42:30.376 INFO 12717 --- [host-check-agent-status1,ae80993d3579a031,97681c07f3448214] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:42:30.377 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:42:30.377 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:42:30.377 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:42:30.378 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:42:30.379 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:42:30.380 INFO 12717 --- [host-check-agent-status1,ae80993d3579a031,97681c07f3448214] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:42:30.381 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:42:30.385 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:42:30.451 INFO 12717 --- [host-check-agent-status4,ae80993d3579a031,3a737440997f9a62] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:42:30.452 INFO 12717 --- [host-check-agent-status10,ae80993d3579a031,ab26817a786fe25e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:42:30.452 INFO 12717 --- [host-check-agent-status5,ae80993d3579a031,dba2b9104e9a5e0e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:42:30.453 INFO 12717 --- [host-check-agent-status3,ae80993d3579a031,75fb3d154728613c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:42:30.454 INFO 12717 --- [host-check-agent-status11,ae80993d3579a031,fe56842de3355579] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:42:30.454 INFO 12717 --- [host-check-agent-status12,ae80993d3579a031,90468e4b30628f3f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:42:30.454 INFO 12717 --- [host-check-agent-status2,ae80993d3579a031,d748c911c6cb2f7b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:42:30.455 INFO 12717 --- [host-check-agent-status8,ae80993d3579a031,dc4b334f1e7dc285] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:42:30.455 INFO 12717 --- [host-check-agent-status9,ae80993d3579a031,75fa515c9adef66e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:42:30.455 INFO 12717 --- [host-check-agent-status6,ae80993d3579a031,e3dd83b3e0a3fbd3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:42:30.456 INFO 12717 --- [host-check-agent-status7,ae80993d3579a031,37b0db692df7e393] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:42:30.458 INFO 12717 --- [host-check-agent-status4,ae80993d3579a031,3a737440997f9a62] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:42:30.458 INFO 12717 --- [host-check-agent-status10,ae80993d3579a031,ab26817a786fe25e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:42:30.458 INFO 12717 --- [host-check-agent-status5,ae80993d3579a031,dba2b9104e9a5e0e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:42:30.459 INFO 12717 --- [host-check-agent-status3,ae80993d3579a031,75fb3d154728613c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:42:30.459 INFO 12717 --- [host-check-agent-status12,ae80993d3579a031,90468e4b30628f3f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:42:30.459 INFO 12717 --- [host-check-agent-status11,ae80993d3579a031,fe56842de3355579] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:42:30.460 INFO 12717 --- [host-check-agent-status2,ae80993d3579a031,d748c911c6cb2f7b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:42:30.461 INFO 12717 --- [host-check-agent-status9,ae80993d3579a031,75fa515c9adef66e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:42:30.461 INFO 12717 --- [host-check-agent-status8,ae80993d3579a031,dc4b334f1e7dc285] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:42:30.461 INFO 12717 --- [host-check-agent-status6,ae80993d3579a031,e3dd83b3e0a3fbd3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:42:30.461 INFO 12717 --- [host-check-agent-status7,ae80993d3579a031,37b0db692df7e393] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:42:30.463 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:30.464 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:42:30.465 INFO 12717 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.8 2025-11-29 08:42:30.468 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:30.469 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:42:30.484 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/time, params:null 2025-11-29 08:42:30.486 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/time, params:null 2025-11-29 08:42:30.486 INFO 12717 --- [host-check-agent-status1,ae80993d3579a031,97681c07f3448214] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.486 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/time, params:null 2025-11-29 08:42:30.487 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/time, params:null 2025-11-29 08:42:30.487 INFO 12717 --- [host-check-agent-status5,ae80993d3579a031,dba2b9104e9a5e0e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.487 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/time, params:null 2025-11-29 08:42:30.487 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/time, params:null 2025-11-29 08:42:30.488 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.202.22.58, timestamp=2025-11-29T08:42:30.488008193+08:00 2025-11-29 08:42:30.488 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/time, params:null 2025-11-29 08:42:30.488 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=11, ip=10.202.22.47, timestamp=2025-11-29T08:42:30.487739954+08:00 2025-11-29 08:42:30.489 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-11-29T08:42:30.488+08:00 2025-11-29 08:42:30.489 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.202.22.54, timestamp=2025-11-29T08:42:30.487214317+08:00 2025-11-29 08:42:30.489 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=11, lastAvailableTime=2025-11-29T08:42:30.488+08:00 2025-11-29 08:42:30.489 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/time, params:null 2025-11-29 08:42:30.490 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.202.22.51, timestamp=2025-11-29T08:42:30.487065713+08:00 2025-11-29 08:42:30.490 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2025-11-29T08:42:30.489+08:00 2025-11-29 08:42:30.490 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-11-29T08:42:30.490+08:00 2025-11-29 08:42:30.490 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/time, params:null 2025-11-29 08:42:30.491 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/time, params:null 2025-11-29 08:42:30.491 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=13, ip=10.202.22.49, timestamp=2025-11-29T08:42:30.489173588+08:00 2025-11-29 08:42:30.491 INFO 12717 --- [host-check-agent-status7,ae80993d3579a031,37b0db692df7e393] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.491 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.202.22.50, timestamp=2025-11-29T08:42:30.491806264+08:00 2025-11-29 08:42:30.492 INFO 12717 --- [host-check-agent-status1,ae80993d3579a031,97681c07f3448214] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.58 2025-11-29 08:42:30.492 INFO 12717 --- [host-check-agent-status1,ae80993d3579a031,97681c07f3448214] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.58 2025-11-29 08:42:30.492 INFO 12717 --- [host-check-agent-status1,ae80993d3579a031,97681c07f3448214] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.58 2025-11-29 08:42:30.492 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.s.compute.HostAgentDaoManager : Updated agent 11 last available time 2025-11-29T08:42:30.488+08:00 and status AVAILABLE 2025-11-29 08:42:30.492 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-11-29T08:42:30.489+08:00 and status AVAILABLE 2025-11-29 08:42:30.492 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-11-29T08:42:30.491+08:00 2025-11-29 08:42:30.492 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=13, lastAvailableTime=2025-11-29T08:42:30.491+08:00 2025-11-29 08:42:30.492 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-11-29T08:42:30.488+08:00 and status AVAILABLE 2025-11-29 08:42:30.493 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.202.22.52, timestamp=2025-11-29T08:42:30.490436663+08:00 2025-11-29 08:42:30.493 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/time, params:null 2025-11-29 08:42:30.493 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=12, ip=10.202.22.48, timestamp=2025-11-29T08:42:30.490588212+08:00 2025-11-29 08:42:30.493 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-11-29T08:42:30.490+08:00 and status AVAILABLE 2025-11-29 08:42:30.493 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-11-29T08:42:30.493+08:00 2025-11-29 08:42:30.494 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=12, lastAvailableTime=2025-11-29T08:42:30.493+08:00 2025-11-29 08:42:30.494 INFO 12717 --- [host-check-agent-status12,ae80993d3579a031,90468e4b30628f3f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.495 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/time, params:null 2025-11-29 08:42:30.495 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2025-11-29T08:42:30.491+08:00 and status AVAILABLE 2025-11-29 08:42:30.496 INFO 12717 --- [host-check-agent-status5,ae80993d3579a031,dba2b9104e9a5e0e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.53 2025-11-29 08:42:30.496 INFO 12717 --- [host-check-agent-status5,ae80993d3579a031,dba2b9104e9a5e0e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.53 2025-11-29 08:42:30.496 INFO 12717 --- [host-check-agent-status5,ae80993d3579a031,dba2b9104e9a5e0e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.53 2025-11-29 08:42:30.496 INFO 12717 --- [host-check-agent-status9,ae80993d3579a031,75fa515c9adef66e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.497 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.s.compute.HostAgentDaoManager : Updated agent 12 last available time 2025-11-29T08:42:30.491+08:00 and status AVAILABLE 2025-11-29 08:42:30.497 INFO 12717 --- [host-check-agent-status11,ae80993d3579a031,fe56842de3355579] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.497 INFO 12717 --- [host-check-agent-status2,ae80993d3579a031,d748c911c6cb2f7b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.497 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.202.22.53, timestamp=2025-11-29T08:42:30.496094193+08:00 2025-11-29 08:42:30.497 INFO 12717 --- [host-check-agent-status8,ae80993d3579a031,dc4b334f1e7dc285] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.497 INFO 12717 --- [host-check-agent-status4,ae80993d3579a031,3a737440997f9a62] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.497 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-11-29T08:42:30.493+08:00 and status AVAILABLE 2025-11-29 08:42:30.497 INFO 12717 --- [host-check-agent-status6,ae80993d3579a031,e3dd83b3e0a3fbd3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.498 INFO 12717 --- [host-check-agent-status3,ae80993d3579a031,75fb3d154728613c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.498 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.s.compute.HostAgentDaoManager : Updated agent 13 last available time 2025-11-29T08:42:30.493+08:00 and status AVAILABLE 2025-11-29 08:42:30.498 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-11-29T08:42:30.497+08:00 2025-11-29 08:42:30.498 INFO 12717 --- [host-check-agent-status12,ae80993d3579a031,90468e4b30628f3f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.49 2025-11-29 08:42:30.498 INFO 12717 --- [host-check-agent-status12,ae80993d3579a031,90468e4b30628f3f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.49 2025-11-29 08:42:30.498 INFO 12717 --- [host-check-agent-status12,ae80993d3579a031,90468e4b30628f3f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.49 2025-11-29 08:42:30.500 INFO 12717 --- [host-check-agent-status11,ae80993d3579a031,fe56842de3355579] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.48 2025-11-29 08:42:30.500 INFO 12717 --- [host-check-agent-status11,ae80993d3579a031,fe56842de3355579] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.48 2025-11-29 08:42:30.500 INFO 12717 --- [host-check-agent-status11,ae80993d3579a031,fe56842de3355579] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.48 2025-11-29 08:42:30.500 INFO 12717 --- [host-check-agent-status2,ae80993d3579a031,d748c911c6cb2f7b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.50 2025-11-29 08:42:30.500 INFO 12717 --- [host-check-agent-status2,ae80993d3579a031,d748c911c6cb2f7b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.50 2025-11-29 08:42:30.500 INFO 12717 --- [host-check-agent-status2,ae80993d3579a031,d748c911c6cb2f7b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.50 2025-11-29 08:42:30.501 INFO 12717 --- [host-check-agent-status10,ae80993d3579a031,ab26817a786fe25e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/agent/status, params:null 2025-11-29 08:42:30.501 INFO 12717 --- [host-check-agent-status4,ae80993d3579a031,3a737440997f9a62] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.52 2025-11-29 08:42:30.501 INFO 12717 --- [host-check-agent-status4,ae80993d3579a031,3a737440997f9a62] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.52 2025-11-29 08:42:30.501 INFO 12717 --- [host-check-agent-status4,ae80993d3579a031,3a737440997f9a62] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.52 2025-11-29 08:42:30.503 INFO 12717 --- [host-check-agent-status3,ae80993d3579a031,75fb3d154728613c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.51 2025-11-29 08:42:30.503 INFO 12717 --- [host-check-agent-status3,ae80993d3579a031,75fb3d154728613c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.51 2025-11-29 08:42:30.503 INFO 12717 --- [host-check-agent-status3,ae80993d3579a031,75fb3d154728613c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.51 2025-11-29 08:42:30.504 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-11-29T08:42:30.497+08:00 and status AVAILABLE 2025-11-29 08:42:30.504 INFO 12717 --- [host-check-agent-status6,ae80993d3579a031,e3dd83b3e0a3fbd3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.54 2025-11-29 08:42:30.504 INFO 12717 --- [host-check-agent-status6,ae80993d3579a031,e3dd83b3e0a3fbd3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.54 2025-11-29 08:42:30.504 INFO 12717 --- [host-check-agent-status6,ae80993d3579a031,e3dd83b3e0a3fbd3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.54 2025-11-29 08:42:30.504 INFO 12717 --- [host-check-agent-status10,ae80993d3579a031,ab26817a786fe25e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.47 2025-11-29 08:42:30.504 INFO 12717 --- [host-check-agent-status10,ae80993d3579a031,ab26817a786fe25e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.47 2025-11-29 08:42:30.504 INFO 12717 --- [host-check-agent-status10,ae80993d3579a031,ab26817a786fe25e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.47 2025-11-29 08:42:30.593 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=9, ip=10.202.22.60, timestamp=2025-11-29T08:42:30.587593334+08:00 2025-11-29 08:42:30.595 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=9, lastAvailableTime=2025-11-29T08:42:30.593+08:00 2025-11-29 08:42:30.595 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=10, ip=10.202.22.61, timestamp=2025-11-29T08:42:30.592901435+08:00 2025-11-29 08:42:30.596 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=10, lastAvailableTime=2025-11-29T08:42:30.595+08:00 2025-11-29 08:42:30.597 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.s.compute.HostAgentDaoManager : Updated agent 10 last available time 2025-11-29T08:42:30.593+08:00 and status AVAILABLE 2025-11-29 08:42:30.598 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2025-11-29T08:42:30.595+08:00 and status AVAILABLE 2025-11-29 08:42:30.600 INFO 12717 --- [host-check-agent-status9,ae80993d3579a031,75fa515c9adef66e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.61 2025-11-29 08:42:30.600 INFO 12717 --- [host-check-agent-status9,ae80993d3579a031,75fa515c9adef66e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.61 2025-11-29 08:42:30.600 INFO 12717 --- [host-check-agent-status9,ae80993d3579a031,75fa515c9adef66e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.61 2025-11-29 08:42:30.601 INFO 12717 --- [Check status for all clusters2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.3.5.1 2025-11-29 08:42:30.601 INFO 12717 --- [host-check-agent-status8,ae80993d3579a031,dc4b334f1e7dc285] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.60 2025-11-29 08:42:30.601 INFO 12717 --- [host-check-agent-status8,ae80993d3579a031,dc4b334f1e7dc285] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.60 2025-11-29 08:42:30.601 INFO 12717 --- [host-check-agent-status8,ae80993d3579a031,dc4b334f1e7dc285] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.60 2025-11-29 08:42:30.605 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-11-29 08:42:30.605 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=13 2025-11-29 08:42:30.605 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2025-11-29 08:42:30.605 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-11-29 08:42:30.605 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=11 2025-11-29 08:42:30.605 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=12 2025-11-29 08:42:30.605 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-11-29 08:42:30.605 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-11-29 08:42:30.605 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-11-29 08:42:30.614 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=10 2025-11-29 08:42:30.614 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=9 2025-11-29 08:42:30.639 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:42:30.639 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:42:30.640 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:42:30.640 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:42:30.640 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:42:30.641 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:42:30.641 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:42:30.641 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:42:30.641 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:42:30.641 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:42:30.641 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:42:30.641 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:42:30.641 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:42:30.641 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:42:30.642 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:42:30.642 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:42:30.642 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:42:30.642 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:42:30.643 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:42:30.643 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:42:30.644 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:42:30.644 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:42:30.659 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.660 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.660 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.660 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.660 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.661 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.662 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.663 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.664 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.664 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.664 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.684 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=8, ip=10.202.22.59, timestamp=2025-11-29T08:42:30.683889907+08:00 2025-11-29 08:42:30.685 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=8, lastAvailableTime=2025-11-29T08:42:30.685+08:00 2025-11-29 08:42:30.687 INFO 12717 --- [host-check-agent-status7,ae80993d3579a031,37b0db692df7e393] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.59 2025-11-29 08:42:30.687 INFO 12717 --- [host-check-agent-status7,ae80993d3579a031,37b0db692df7e393] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.59 2025-11-29 08:42:30.687 INFO 12717 --- [host-check-agent-status7,ae80993d3579a031,37b0db692df7e393] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.59 2025-11-29 08:42:30.689 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.s.compute.HostAgentDaoManager : Updated agent 8 last available time 2025-11-29T08:42:30.685+08:00 and status AVAILABLE 2025-11-29 08:42:30.697 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=8 2025-11-29 08:42:30.718 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-11-29 08:42:30.718 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:42:30.720 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-11-29 08:42:30.721 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=16 2025-11-29 08:42:30.721 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=25 2025-11-29 08:42:30.722 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:42:30.729 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=34 2025-11-29 08:42:30.731 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=28 2025-11-29 08:42:30.732 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-11-29 08:42:30.733 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=31 2025-11-29 08:42:30.734 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-11-29 08:42:30.742 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=25 2025-11-29 08:42:30.743 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-11-29 08:42:30.743 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-11-29 08:42:30.744 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-11-29 08:42:30.744 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-11-29 08:42:30.744 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=16 2025-11-29 08:42:30.744 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-11-29 08:42:30.745 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=34 2025-11-29 08:42:30.745 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=28 2025-11-29 08:42:30.749 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=31 2025-11-29 08:42:30.758 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-11-29 08:42:30.759 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:42:30.775 INFO 12717 --- [prometheus-second-schedule-1,9986a41582237f89,5b494e823530865c] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=42, collectTaskCount=42 2025-11-29 08:42:30.812 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=22 2025-11-29 08:42:30.822 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=22 2025-11-29 08:42:30.830 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=37 2025-11-29 08:42:30.839 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=37 2025-11-29 08:42:30.975 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.976 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.978 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.989 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.989 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.989 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.990 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.991 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.991 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.991 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.992 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:30.992 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:42:31.033 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=32 2025-11-29 08:42:31.036 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-11-29 08:42:31.040 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-11-29 08:42:31.041 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=26 2025-11-29 08:42:31.043 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=32 2025-11-29 08:42:31.046 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-11-29 08:42:31.051 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-11-29 08:42:31.052 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-11-29 08:42:31.052 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=26 2025-11-29 08:42:31.052 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=17 2025-11-29 08:42:31.056 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-11-29 08:42:31.059 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=35 2025-11-29 08:42:31.063 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-11-29 08:42:31.064 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=17 2025-11-29 08:42:31.065 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=29 2025-11-29 08:42:31.066 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-11-29 08:42:31.066 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.069 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.069 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=35 2025-11-29 08:42:31.070 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-11-29 08:42:31.076 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=29 2025-11-29 08:42:31.078 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.081 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.083 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-11-29 08:42:31.084 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.087 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.089 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.093 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.099 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=23 2025-11-29 08:42:31.100 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.106 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.110 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=23 2025-11-29 08:42:31.124 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-11-29 08:42:31.125 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=33 2025-11-29 08:42:31.130 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.132 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-11-29 08:42:31.135 INFO 12717 --- [host-check-host-status5,f0b6d6db80f198aa,f778d16634c22326] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-11-29 08:42:31.136 INFO 12717 --- [host-check-host-status10,f0b6d6db80f198aa,5f4750226ce53b2a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=33 2025-11-29 08:42:31.140 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=18 2025-11-29 08:42:31.143 INFO 12717 --- [host-check-host-status6,f0b6d6db80f198aa,673180b07ef94a18] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-11-29 08:42:31.146 WARN 12717 --- [host-check-clockdiff10,2d6619c1f8c43c40,752fb5985b0a3025] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.47], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.146 ERROR 12717 --- [host-check-clockdiff10,2d6619c1f8c43c40,752fb5985b0a3025] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.47] 2025-11-29 08:42:31.147 WARN 12717 --- [host-check-clockdiff10,2d6619c1f8c43c40,752fb5985b0a3025] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=11, ip=10.202.22.47, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.47 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.150 INFO 12717 --- [host-check-host-status2,f0b6d6db80f198aa,cef8bb21c290fcd8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=18 2025-11-29 08:42:31.154 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=36 2025-11-29 08:42:31.154 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-11-29 08:42:31.156 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-11-29 08:42:31.160 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-11-29 08:42:31.163 INFO 12717 --- [host-check-host-status12,f0b6d6db80f198aa,da4b477f4017e2c2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=36 2025-11-29 08:42:31.163 INFO 12717 --- [host-check-host-status4,f0b6d6db80f198aa,8f27ccf97b87688c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-11-29 08:42:31.165 INFO 12717 --- [host-check-host-status3,f0b6d6db80f198aa,cf2c68affaf32d78] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-11-29 08:42:31.168 INFO 12717 --- [host-check-host-status1,f0b6d6db80f198aa,4f65b19304247556] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-11-29 08:42:31.187 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=24 2025-11-29 08:42:31.195 INFO 12717 --- [host-check-host-status7,f0b6d6db80f198aa,6cebaab34cff6dfc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=24 2025-11-29 08:42:31.228 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=38 2025-11-29 08:42:31.237 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=38 2025-11-29 08:42:31.336 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:42:31.353 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=27 2025-11-29 08:42:31.364 INFO 12717 --- [host-check-host-status9,f0b6d6db80f198aa,baf74c810ed56a9f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=27 2025-11-29 08:42:31.377 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=30 2025-11-29 08:42:31.387 INFO 12717 --- [host-check-host-status8,f0b6d6db80f198aa,d9b38d7cae0e7da2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=30 2025-11-29 08:42:31.403 WARN 12717 --- [host-check-clockdiff12,2d6619c1f8c43c40,b17e06bb099d18bc] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.49], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.403 ERROR 12717 --- [host-check-clockdiff12,2d6619c1f8c43c40,b17e06bb099d18bc] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.49] 2025-11-29 08:42:31.403 WARN 12717 --- [host-check-clockdiff12,2d6619c1f8c43c40,b17e06bb099d18bc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=13, ip=10.202.22.49, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.49 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.478 WARN 12717 --- [host-check-clockdiff4,2d6619c1f8c43c40,251b49cdb8b28bed] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.52], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.478 ERROR 12717 --- [host-check-clockdiff4,2d6619c1f8c43c40,251b49cdb8b28bed] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.52] 2025-11-29 08:42:31.478 WARN 12717 --- [host-check-clockdiff4,2d6619c1f8c43c40,251b49cdb8b28bed] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.202.22.52, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.52 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.480 WARN 12717 --- [host-check-clockdiff6,2d6619c1f8c43c40,23ce851b5783ea1b] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.54], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.480 ERROR 12717 --- [host-check-clockdiff6,2d6619c1f8c43c40,23ce851b5783ea1b] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.54] 2025-11-29 08:42:31.480 WARN 12717 --- [host-check-clockdiff6,2d6619c1f8c43c40,23ce851b5783ea1b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.202.22.54, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.54 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.488 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:31.489 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:31.607 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=39 2025-11-29 08:42:31.617 INFO 12717 --- [host-check-host-status11,f0b6d6db80f198aa,0d04719eca139224] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=39 2025-11-29 08:42:31.679 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,425c47adbe9505a5,ccaf3fa839719fdc] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=425c47adbe9505a5, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:31.685 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,425c47adbe9505a5,ccaf3fa839719fdc] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=425c47adbe9505a5, duration=7 ms] 2025-11-29 08:42:31.690 INFO 12717 --- [ocp-async-32,5095307e6acd755c,2ba9096b6bb90e95] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:31.814 WARN 12717 --- [host-check-clockdiff2,2d6619c1f8c43c40,dd7936ff04371204] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.50], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.814 ERROR 12717 --- [host-check-clockdiff2,2d6619c1f8c43c40,dd7936ff04371204] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.50] 2025-11-29 08:42:31.814 WARN 12717 --- [host-check-clockdiff2,2d6619c1f8c43c40,dd7936ff04371204] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.202.22.50, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.50 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.837 WARN 12717 --- [host-check-clockdiff5,2d6619c1f8c43c40,95b5af83df7db142] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.53], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.837 ERROR 12717 --- [host-check-clockdiff5,2d6619c1f8c43c40,95b5af83df7db142] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.53] 2025-11-29 08:42:31.837 WARN 12717 --- [host-check-clockdiff5,2d6619c1f8c43c40,95b5af83df7db142] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.202.22.53, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.53 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.859 WARN 12717 --- [host-check-clockdiff3,2d6619c1f8c43c40,e310f235d44e7ba0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.51], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:31.859 ERROR 12717 --- [host-check-clockdiff3,2d6619c1f8c43c40,e310f235d44e7ba0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.51] 2025-11-29 08:42:31.859 WARN 12717 --- [host-check-clockdiff3,2d6619c1f8c43c40,e310f235d44e7ba0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.202.22.51, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.51 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:32.171 INFO 12717 --- [ocp-schedules-21,13d3172bd218e3b4,69b4262672bfe844] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:42:32.192 WARN 12717 --- [scanner-11213810885-1,13d3172bd218e3b4,2693d5f6fa143e65] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:42:32.407 INFO 12717 --- [prometheus-minute-schedule-1,499c0b71779edfad,ec18a42805a5d09e] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:32.414 INFO 12717 --- [prometheus-minute-schedule-1,499c0b71779edfad,ec18a42805a5d09e] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:32.752 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-77,f4aa19291cb20b0a,929e96f2bf965fdf] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=f4aa19291cb20b0a, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:32.753 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-77,f4aa19291cb20b0a,929e96f2bf965fdf] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=f4aa19291cb20b0a, duration=1 ms] 2025-11-29 08:42:32.829 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-100,a1e88858cd96960a,e39c6768a8e6429e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=a1e88858cd96960a, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:32.830 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-100,a1e88858cd96960a,e39c6768a8e6429e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=a1e88858cd96960a, duration=2 ms] 2025-11-29 08:42:33.065 WARN 12717 --- [host-check-clockdiff11,2d6619c1f8c43c40,45ac1eb191929386] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.48], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:33.065 ERROR 12717 --- [host-check-clockdiff11,2d6619c1f8c43c40,45ac1eb191929386] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.48] 2025-11-29 08:42:33.065 WARN 12717 --- [host-check-clockdiff11,2d6619c1f8c43c40,45ac1eb191929386] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=12, ip=10.202.22.48, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.48 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:33.129 INFO 12717 --- [ocp-async-1,407694d0af33c804,4f65fd70dc0c23c7] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:42:33.323 INFO 12717 --- [host-check-clockdiff9,2d6619c1f8c43c40,7a2af28fb30f9e18] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376953 -2 -2 , error= 2025-11-29 08:42:33.536 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-98,33dd8c44e7147fa9,cc83272db13464ca] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=33dd8c44e7147fa9, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:33.536 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-98,33dd8c44e7147fa9,cc83272db13464ca] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=33dd8c44e7147fa9, duration=1 ms] 2025-11-29 08:42:33.690 INFO 12717 --- [ocp-async-28,9cbf33b8ae528a4d,97cacfe84a754cc5] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:33.814 INFO 12717 --- [ocp-schedules-18,89ea9788365f6f6a,3dbca4b047ee1df3] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:42:33.814 INFO 12717 --- [ocp-schedules-18,89ea9788365f6f6a,3dbca4b047ee1df3] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=9 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=12 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=10 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=8 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-29,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=6 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=13 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=11 2025-11-29 08:42:33.814 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2025-11-29 08:42:33.821 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:42:33.821 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:42:33.821 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:42:33.821 INFO 12717 --- [ForkJoinPool.commonPool-worker-29,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:42:33.821 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:42:33.821 INFO 12717 --- [ocp-schedules-18,89ea9788365f6f6a,3dbca4b047ee1df3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:42:33.821 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:42:33.821 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:42:33.821 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:42:33.821 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:42:33.822 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:42:33.822 INFO 12717 --- [ForkJoinPool.commonPool-worker-29,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:42:33.822 INFO 12717 --- [ocp-schedules-18,89ea9788365f6f6a,3dbca4b047ee1df3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:42:33.822 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:42:33.822 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:42:33.823 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:42:33.823 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:42:33.823 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:42:33.823 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:42:33.823 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:42:33.823 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:42:33.824 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:42:33.824 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:42:33.825 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:42:33.844 INFO 12717 --- [ocp-schedules-18,89ea9788365f6f6a,3dbca4b047ee1df3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.845 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.845 INFO 12717 --- [ForkJoinPool.commonPool-worker-29,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.846 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.847 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.847 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.847 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.847 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.848 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.848 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.849 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.850 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62889/api/v1/log/alarms, params:null 2025-11-29 08:42:33.851 WARN 12717 --- [host-check-clockdiff7,2d6619c1f8c43c40,138532199869dbc9] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.59], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:33.851 ERROR 12717 --- [host-check-clockdiff7,2d6619c1f8c43c40,138532199869dbc9] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.59] 2025-11-29 08:42:33.851 WARN 12717 --- [host-check-clockdiff7,2d6619c1f8c43c40,138532199869dbc9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=8, ip=10.202.22.59, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.59 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:42:33.923 INFO 12717 --- [ocp-schedules-18,89ea9788365f6f6a,3dbca4b047ee1df3] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms finished. 2025-11-29 08:42:33.927 INFO 12717 --- [ocp-schedules-25,68872e369144c252,f3941c0f0550d846] c.o.o.s.c.scheduled.TenantSchedules : Sync all tenant info schedule. 2025-11-29 08:42:33.929 INFO 12717 --- [ocp-schedules-25,68872e369144c252,f3941c0f0550d846] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:42:33.964 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-11-29 08:42:33.964 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:33.968 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-11-29 08:42:33.975 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:33.975 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-11-29 08:42:33.975 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:33.976 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-11-29 08:42:33.979 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-11-29 08:42:33.985 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:33.985 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-11-29 08:42:33.994 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:33.994 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2025-11-29 08:42:34.004 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.005 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2025-11-29 08:42:34.015 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.015 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1004] 2025-11-29 08:42:34.024 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.025 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-11-29 08:42:34.028 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.028 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1004] 2025-11-29 08:42:34.046 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.047 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2025-11-29 08:42:34.047 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.048 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:42:34.049 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.049 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:42:34.050 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.051 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2025-11-29 08:42:34.052 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.052 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:42:34.053 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.054 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:42:34.054 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.055 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1004, ALTER_TENANT_LOCALITY] 2025-11-29 08:42:34.056 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.056 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1004, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:42:34.057 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.057 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.057 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-11-29 08:42:34.058 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1004, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:42:34.058 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2025-11-29 08:42:34.063 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.202.22.58 2025-11-29 08:42:34.077 INFO 12717 --- [Sync all tenant information1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:34.078 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.078 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE name = ?, args: [ob_tcp_invited_nodes] 2025-11-29 08:42:34.085 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:42:34.085 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2025-11-29 08:42:34.085 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=3, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:42:34.085 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=3, userWhitelist=% 2025-11-29 08:42:34.085 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:42:34.085 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2025-11-29 08:42:34.089 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2025-11-29 08:42:34.090 INFO 12717 --- [Sync all tenant information1,,] .o.s.t.b.t.SyncAllTenantInfoShardingTask : [OCP606] sync tenants success, clusterId=1, clusterName=ocpserver, obClusterId=1743215593 2025-11-29 08:42:34.093 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.094 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1] 2025-11-29 08:42:34.097 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.097 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1010] 2025-11-29 08:42:34.129 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.130 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1010] 2025-11-29 08:42:34.164 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.164 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1010] 2025-11-29 08:42:34.168 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.168 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1014] 2025-11-29 08:42:34.200 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.201 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1014] 2025-11-29 08:42:34.234 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.235 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1014] 2025-11-29 08:42:34.241 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.241 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1018] 2025-11-29 08:42:34.276 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.277 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1018] 2025-11-29 08:42:34.314 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.314 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1018] 2025-11-29 08:42:34.317 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.318 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1020] 2025-11-29 08:42:34.349 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.349 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1020] 2025-11-29 08:42:34.380 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.381 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1020] 2025-11-29 08:42:34.383 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.383 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1022] 2025-11-29 08:42:34.414 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.415 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1022] 2025-11-29 08:42:34.446 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.447 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1022] 2025-11-29 08:42:34.453 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.454 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1024] 2025-11-29 08:42:34.486 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.487 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1024] 2025-11-29 08:42:34.489 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:34.491 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:34.525 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.526 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1024] 2025-11-29 08:42:34.568 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.569 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2025-11-29 08:42:34.571 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.572 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:42:34.574 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.575 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:42:34.578 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.579 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1010, ALTER_TENANT_LOCALITY] 2025-11-29 08:42:34.582 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.583 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1010, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:42:34.585 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.586 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1010, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:42:34.588 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.589 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1014, ALTER_TENANT_LOCALITY] 2025-11-29 08:42:34.591 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.592 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1014, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:42:34.594 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.595 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1014, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:42:34.598 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.599 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1018, ALTER_TENANT_LOCALITY] 2025-11-29 08:42:34.602 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.603 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1018, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:42:34.605 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.606 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1018, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:42:34.608 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.609 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1020, ALTER_TENANT_LOCALITY] 2025-11-29 08:42:34.612 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.612 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1020, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:42:34.614 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.615 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1020, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:42:34.617 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.618 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1022, ALTER_TENANT_LOCALITY] 2025-11-29 08:42:34.620 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.621 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1022, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:42:34.623 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.623 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1022, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:42:34.625 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.626 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1024, ALTER_TENANT_LOCALITY] 2025-11-29 08:42:34.628 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.629 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1024, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:42:34.631 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.632 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1024, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:42:34.634 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=2 2025-11-29 08:42:34.639 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.202.22.58 2025-11-29 08:42:34.668 INFO 12717 --- [Sync all tenant information2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:34.668 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:34.669 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE name = ?, args: [ob_tcp_invited_nodes] 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=9, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=9, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=13, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=13, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=11, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=11, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=14, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=14, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=4, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=4, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=16, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=16, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=15, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:42:34.717 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=15, userWhitelist=% 2025-11-29 08:42:34.724 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=2 2025-11-29 08:42:34.724 INFO 12717 --- [Sync all tenant information2,,] .o.s.t.b.t.SyncAllTenantInfoShardingTask : [OCP606] sync tenants success, clusterId=2, clusterName=DB, obClusterId=1743215594 2025-11-29 08:42:35.407 INFO 12717 --- [prometheus-minute-schedule-1,574ca241168af230,a4a6daa0682a31ec] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:35.418 INFO 12717 --- [prometheus-minute-schedule-1,574ca241168af230,a4a6daa0682a31ec] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:35.690 INFO 12717 --- [ocp-async-28,8a7a83fceefbb028,d3ddda4235bee8c1] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:36.596 INFO 12717 --- [pool-18-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : Starting PLAN_CHANGE_DETECTOR, Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591} 2025-11-29 08:42:36.606 INFO 12717 --- [pool-18-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR : for [Pair(left=Tenant(id=15, name=zto, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-06-19T19:31:51+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1022, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Pair(left=Tenant(id=3, name=ocp_monitor, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1004, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Pair(left=Tenant(id=13, name=nextcloud, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T13:13:08+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1018, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTe 2025-11-29 08:42:36.607 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:42:36.614 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=15, name=zto, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-06-19T19:31:51+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1022, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:42:36.614 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=15, name=zto, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-06-19T19:31:51+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1022, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null) 2025-11-29 08:42:36.615 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:42:36.896 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290556896000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, sqlId=858015B92843B43D99CA77541C4C2592, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=15, obTenantId=1022, tenantName=zto, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:36.896Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=15, name=zto, obTenantId=1022, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:36.896 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290556896000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, sqlId=AD63995D4A6E8C3787D81BDA88E06774, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=15, obTenantId=1022, tenantName=zto, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:36.896Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=15, name=zto, obTenantId=1022, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:36.899 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290556899000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, sqlId=5BD7065651E80825844BA74A47825BF8, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=15, obTenantId=1022, tenantName=zto, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:36.899Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=15, name=zto, obTenantId=1022, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:36.919 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:36.919 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:36.919 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:36.919 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:36.933 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:36.933 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=201001, sqlId=AD63995D4A6E8C3787D81BDA88E06774, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:36.934 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:36.934 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=201001, sqlId=5BD7065651E80825844BA74A47825BF8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:36.941 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:36.941 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:36.943 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:36.943 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:36.945 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1521 2025-11-29 08:42:36.945 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:36.958 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:36.958 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=201001, sqlId=5BD7065651E80825844BA74A47825BF8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:36.959 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:36.959 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=201001, sqlId=AD63995D4A6E8C3787D81BDA88E06774, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:36.960 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:36.960 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:36.961 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:36.962 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:36.965 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:36.965 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:36.975 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:36.896, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=15, sqlId=AD63995D4A6E8C3787D81BDA88E06774, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:36.975 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:36.899, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=15, sqlId=5BD7065651E80825844BA74A47825BF8, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:36.981 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:36.981 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:36.984 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:42:37.001 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:36.896, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=15, sqlId=858015B92843B43D99CA77541C4C2592, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.001 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:42:37.003 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=3, name=ocp_monitor, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1004, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:42:37.003 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=3, name=ocp_monitor, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1004, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null) 2025-11-29 08:42:37.003 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:42:37.005 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:42:37.007 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=13, name=nextcloud, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T13:13:08+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1018, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:42:37.007 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=13, name=nextcloud, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T13:13:08+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1018, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null) 2025-11-29 08:42:37.007 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:42:37.050 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557050000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, sqlId=34DE9423E43C494DD78B70EE21469C99, obServerId=null, obDbId=500645, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=13, obTenantId=1018, tenantName=nextcloud, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.050Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=13, name=nextcloud, obTenantId=1018, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPo 2025-11-29 08:42:37.051 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557051000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, sqlId=32956A9F60CD705D45AAE8787A7C480B, obServerId=null, obDbId=500645, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=13, obTenantId=1018, tenantName=nextcloud, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.051Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=13, name=nextcloud, obTenantId=1018, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPo 2025-11-29 08:42:37.052 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557052000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, sqlId=40B90AD1A3265F27229BE2B0BBAF4165, obServerId=null, obDbId=500645, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=13, obTenantId=1018, tenantName=nextcloud, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.052Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=13, name=nextcloud, obTenantId=1018, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPo 2025-11-29 08:42:37.054 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557054000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, sqlId=A3E0EB2AC80AFA1556AC5DFCD8750E0B, obServerId=null, obDbId=500645, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=13, obTenantId=1018, tenantName=nextcloud, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.054Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=13, name=nextcloud, obTenantId=1018, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPo 2025-11-29 08:42:37.205 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 23 2025-11-29 08:42:37.205 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.205 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 23 2025-11-29 08:42:37.205 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.219 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:37.219 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.220 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:37.220 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.221 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.221 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, obServerId=null, obDbId=500645, sqlId=32956A9F60CD705D45AAE8787A7C480B, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.221 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.221 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, obServerId=null, obDbId=500645, sqlId=34DE9423E43C494DD78B70EE21469C99, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.230 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.230 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.230 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.230 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.233 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.233 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, obServerId=null, obDbId=500645, sqlId=40B90AD1A3265F27229BE2B0BBAF4165, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.233 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.233 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, obServerId=null, obDbId=500645, sqlId=A3E0EB2AC80AFA1556AC5DFCD8750E0B, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.235 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.235 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.237 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.237 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.244 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557244000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, sqlId=179EBD99BAF184D4ABE6BBB9C908CB8A, obServerId=null, obDbId=500645, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=13, obTenantId=1018, tenantName=nextcloud, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.244Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=13, name=nextcloud, obTenantId=1018, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPo 2025-11-29 08:42:37.245 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.245 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, obServerId=null, obDbId=500645, sqlId=32956A9F60CD705D45AAE8787A7C480B, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.245 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.245 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, obServerId=null, obDbId=500645, sqlId=34DE9423E43C494DD78B70EE21469C99, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.247 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.247 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.251 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.251 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, obServerId=null, obDbId=500645, sqlId=40B90AD1A3265F27229BE2B0BBAF4165, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.253 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.253 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, obServerId=null, obDbId=500645, sqlId=A3E0EB2AC80AFA1556AC5DFCD8750E0B, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.255 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.255 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.05, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=13, sqlId=34DE9423E43C494DD78B70EE21469C99, serverId=null, dbName=nextcloud, obDbId=500645, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.256 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.051, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=13, sqlId=32956A9F60CD705D45AAE8787A7C480B, serverId=null, dbName=nextcloud, obDbId=500645, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.256 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.256 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:37.256 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.260 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.052, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=13, sqlId=40B90AD1A3265F27229BE2B0BBAF4165, serverId=null, dbName=nextcloud, obDbId=500645, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.263 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.054, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=13, sqlId=A3E0EB2AC80AFA1556AC5DFCD8750E0B, serverId=null, dbName=nextcloud, obDbId=500645, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.273 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.273 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, obServerId=null, obDbId=500645, sqlId=179EBD99BAF184D4ABE6BBB9C908CB8A, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.275 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.275 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.290 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.290 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1018, obServerId=null, obDbId=500645, sqlId=179EBD99BAF184D4ABE6BBB9C908CB8A, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.293 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.297 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.244, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=13, sqlId=179EBD99BAF184D4ABE6BBB9C908CB8A, serverId=null, dbName=nextcloud, obDbId=500645, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.297 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:42:37.302 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=14, name=pipeline, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-29T17:10:07+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1020, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=COMPLEX_OLTP), right=null), Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:42:37.302 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=14, name=pipeline, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-29T17:10:07+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1020, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=COMPLEX_OLTP), right=null) 2025-11-29 08:42:37.302 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:42:37.337 INFO 12717 --- [host-check-clockdiff8,2d6619c1f8c43c40,ef8fbb32494a5ad3] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376957 -5 -5 , error= 2025-11-29 08:42:37.345 INFO 12717 --- [ocp-schedules-30,77567f3e9354762e,1ffb3b043acd2c21] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:42:37.348 INFO 12717 --- [ocp-schedules-30,77567f3e9354762e,1ffb3b043acd2c21] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:42:37.371 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557371000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, sqlId=858015B92843B43D99CA77541C4C2592, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=14, obTenantId=1020, tenantName=pipeline, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.371Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=14, name=pipeline, obTenantId=1020, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort 2025-11-29 08:42:37.372 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557372000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, sqlId=AD63995D4A6E8C3787D81BDA88E06774, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=14, obTenantId=1020, tenantName=pipeline, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.372Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=14, name=pipeline, obTenantId=1020, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort 2025-11-29 08:42:37.372 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557372000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, sqlId=5BD7065651E80825844BA74A47825BF8, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=14, obTenantId=1020, tenantName=pipeline, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.372Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=14, name=pipeline, obTenantId=1020, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort 2025-11-29 08:42:37.373 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557373000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=14, obTenantId=1020, tenantName=pipeline, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.373Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=14, name=pipeline, obTenantId=1020, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort 2025-11-29 08:42:37.386 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 260 2025-11-29 08:42:37.386 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.387 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 260 2025-11-29 08:42:37.387 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.393 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 260 2025-11-29 08:42:37.393 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.400 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.400 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, obServerId=null, obDbId=201001, sqlId=5BD7065651E80825844BA74A47825BF8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.400 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.400 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, obServerId=null, obDbId=201001, sqlId=AD63995D4A6E8C3787D81BDA88E06774, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.402 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.402 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.403 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.403 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.404 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1053 2025-11-29 08:42:37.404 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.407 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.407 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, obServerId=null, obDbId=201001, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.410 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.410 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.415 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.415 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, obServerId=null, obDbId=201001, sqlId=5BD7065651E80825844BA74A47825BF8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.415 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.415 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, obServerId=null, obDbId=201001, sqlId=AD63995D4A6E8C3787D81BDA88E06774, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.417 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.417 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.417 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.418 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.423 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.423 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.424 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.372, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=14, sqlId=AD63995D4A6E8C3787D81BDA88E06774, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.424 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.424 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, obServerId=null, obDbId=201001, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.424 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.372, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=14, sqlId=5BD7065651E80825844BA74A47825BF8, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.426 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.437 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.438 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.441 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:42:37.454 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.371, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=14, sqlId=858015B92843B43D99CA77541C4C2592, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.492 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:37.492 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:37.502 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,01591006bc5c0180,5efab80571b7c46f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=01591006bc5c0180, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:37.503 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,01591006bc5c0180,5efab80571b7c46f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=01591006bc5c0180, duration=1 ms] 2025-11-29 08:42:37.576 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.373, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=14, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.576 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:42:37.580 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:42:37.580 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null) 2025-11-29 08:42:37.580 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:42:37.643 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557643000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, sqlId=858015B92843B43D99CA77541C4C2592, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=9, obTenantId=1010, tenantName=lhp6796, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.643Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=9, name=lhp6796, obTenantId=1010, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=288 2025-11-29 08:42:37.643 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557643000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=9, obTenantId=1010, tenantName=lhp6796, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.643Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=9, name=lhp6796, obTenantId=1010, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=288 2025-11-29 08:42:37.644 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557643000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, sqlId=6FF93F6AA5BE8A3A69BD6DEB867CC7B1, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=9, obTenantId=1010, tenantName=lhp6796, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.643Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=9, name=lhp6796, obTenantId=1010, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=288 2025-11-29 08:42:37.644 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557644000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, sqlId=5BD7065651E80825844BA74A47825BF8, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=9, obTenantId=1010, tenantName=lhp6796, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.644Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=9, name=lhp6796, obTenantId=1010, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=288 2025-11-29 08:42:37.645 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557645000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, sqlId=AD63995D4A6E8C3787D81BDA88E06774, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=9, obTenantId=1010, tenantName=lhp6796, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.645Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=9, name=lhp6796, obTenantId=1010, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=288 2025-11-29 08:42:37.657 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:37.657 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.658 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:37.658 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.666 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:37.666 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.671 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.671 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=5BD7065651E80825844BA74A47825BF8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.671 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.671 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.674 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.674 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.674 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.674 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.675 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1047 2025-11-29 08:42:37.675 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.681 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.681 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=AD63995D4A6E8C3787D81BDA88E06774, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.682 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1046 2025-11-29 08:42:37.682 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.684 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.684 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.688 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.688 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=5BD7065651E80825844BA74A47825BF8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.689 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.689 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.689 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.689 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.690 INFO 12717 --- [ocp-async-31,d8047d18267ff953,b1f9a223459d106d] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:37.691 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.692 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.696 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.696 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.698 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.698 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=6FF93F6AA5BE8A3A69BD6DEB867CC7B1, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.699 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.644, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=9, sqlId=5BD7065651E80825844BA74A47825BF8, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.699 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.643, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=9, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.700 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.700 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=AD63995D4A6E8C3787D81BDA88E06774, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.702 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.702 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.703 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.710 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.710 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.711 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.645, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=9, sqlId=AD63995D4A6E8C3787D81BDA88E06774, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.714 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:42:37.718 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.718 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=6FF93F6AA5BE8A3A69BD6DEB867CC7B1, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.720 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:42:37.727 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.643, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=9, sqlId=858015B92843B43D99CA77541C4C2592, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.732 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.643, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=9, sqlId=6FF93F6AA5BE8A3A69BD6DEB867CC7B1, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.732 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:42:37.735 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=16, name=wecom_bot, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-27T13:42:13+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1024, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:42:37.735 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=16, name=wecom_bot, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-27T13:42:13+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1024, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null) 2025-11-29 08:42:37.735 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:42:37.789 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557789000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, sqlId=858015B92843B43D99CA77541C4C2592, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=16, obTenantId=1024, tenantName=wecom_bot, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.789Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=16, name=wecom_bot, obTenantId=1024, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPo 2025-11-29 08:42:37.790 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557790000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, sqlId=5BD7065651E80825844BA74A47825BF8, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=16, obTenantId=1024, tenantName=wecom_bot, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.790Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=16, name=wecom_bot, obTenantId=1024, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPo 2025-11-29 08:42:37.792 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557792000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=16, obTenantId=1024, tenantName=wecom_bot, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.792Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=16, name=wecom_bot, obTenantId=1024, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPo 2025-11-29 08:42:37.794 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557794000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, sqlId=AD63995D4A6E8C3787D81BDA88E06774, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=16, obTenantId=1024, tenantName=wecom_bot, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.794Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=16, name=wecom_bot, obTenantId=1024, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPo 2025-11-29 08:42:37.803 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:37.803 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.803 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:37.803 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.809 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:37.809 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.816 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1043 2025-11-29 08:42:37.817 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.817 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.817 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, obServerId=null, obDbId=201001, sqlId=5BD7065651E80825844BA74A47825BF8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.818 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.818 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, obServerId=null, obDbId=201001, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.820 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.820 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.820 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.820 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.823 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.823 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, obServerId=null, obDbId=201001, sqlId=AD63995D4A6E8C3787D81BDA88E06774, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.826 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.826 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.832 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.832 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.833 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.833 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, obServerId=null, obDbId=201001, sqlId=5BD7065651E80825844BA74A47825BF8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.835 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.835 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, obServerId=null, obDbId=201001, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.836 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.837 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.837 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.837 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.839 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.839 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, obServerId=null, obDbId=201001, sqlId=AD63995D4A6E8C3787D81BDA88E06774, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.842 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:37.843 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.792, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=16, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.847 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.794, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=16, sqlId=AD63995D4A6E8C3787D81BDA88E06774, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.851 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.851 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.854 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:42:37.859 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.79, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=16, sqlId=5BD7065651E80825844BA74A47825BF8, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.866 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.789, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=16, sqlId=858015B92843B43D99CA77541C4C2592, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:37.866 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:42:37.869 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=11, name=oms, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T10:08:31+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1014, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:42:37.869 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=11, name=oms, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T10:08:31+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1014, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null) 2025-11-29 08:42:37.869 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:42:37.925 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557925000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=CB73A583C8E0474BC3A6A5C4578F229D, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.925Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:37.958 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 263 2025-11-29 08:42:37.958 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.973 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.973 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=CB73A583C8E0474BC3A6A5C4578F229D, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:37.978 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:37.978 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:37.981 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557981000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=5BD7065651E80825844BA74A47825BF8, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.981Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:37.981 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557981000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=5B16335FD7B1D26B67825C0F995123BE, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.981Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:37.982 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557982000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=6C06063F61D7F82604EA25EE26C107F5, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.982Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:37.982 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557982000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=83EDD2B9C15738574DC917C1A9FF23C8, obServerId=null, obDbId=500003, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.982Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:37.983 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557983000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=858015B92843B43D99CA77541C4C2592, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.983Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:37.984 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557984000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=9168B12FC00A9008C84CE9A336E27A96, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.984Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:37.991 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290557991000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=3C6FBF2D06EE17EDCCAB7E9E70ED3711, obServerId=null, obDbId=500003, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:37.991Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:37.991 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:37.991 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:37.993 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:37.993 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=CB73A583C8E0474BC3A6A5C4578F229D, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.000 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 537 2025-11-29 08:42:38.000 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.002 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:38.002 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.004 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.004 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=201001, sqlId=5BD7065651E80825844BA74A47825BF8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.009 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 699 2025-11-29 08:42:38.009 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.010 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 266 2025-11-29 08:42:38.010 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.015 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.015 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=5B16335FD7B1D26B67825C0F995123BE, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.016 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:38.017 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.017 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500003, sqlId=83EDD2B9C15738574DC917C1A9FF23C8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.017 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.017 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.017 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:38.017 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.019 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.019 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.020 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.020 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.024 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.024 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=6C06063F61D7F82604EA25EE26C107F5, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.025 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.925, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=11, sqlId=CB73A583C8E0474BC3A6A5C4578F229D, serverId=null, dbName=_cm, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.025 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.025 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=9168B12FC00A9008C84CE9A336E27A96, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.028 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.028 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.029 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.029 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.031 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.031 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=201001, sqlId=5BD7065651E80825844BA74A47825BF8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.032 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1199 2025-11-29 08:42:38.032 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.032 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.032 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500003, sqlId=3C6FBF2D06EE17EDCCAB7E9E70ED3711, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.034 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:38.034 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.034 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=5B16335FD7B1D26B67825C0F995123BE, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.035 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.035 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.036 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.036 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500003, sqlId=83EDD2B9C15738574DC917C1A9FF23C8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.036 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:42:38.038 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:38.040 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.981, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=11, sqlId=5BD7065651E80825844BA74A47825BF8, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.043 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.043 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=6C06063F61D7F82604EA25EE26C107F5, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.044 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.982, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=11, sqlId=83EDD2B9C15738574DC917C1A9FF23C8, serverId=null, dbName=_rm, obDbId=500003, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.045 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.045 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=9168B12FC00A9008C84CE9A336E27A96, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.046 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:42:38.047 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:38.049 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.049 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.050 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.981, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=11, sqlId=5B16335FD7B1D26B67825C0F995123BE, serverId=null, dbName=_cm, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.052 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.052 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500003, sqlId=3C6FBF2D06EE17EDCCAB7E9E70ED3711, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.053 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.984, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=11, sqlId=9168B12FC00A9008C84CE9A336E27A96, serverId=null, dbName=_cm, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.053 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.053 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.054 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:38.059 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.991, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=11, sqlId=3C6FBF2D06EE17EDCCAB7E9E70ED3711, serverId=null, dbName=_rm, obDbId=500003, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.061 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.982, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=11, sqlId=6C06063F61D7F82604EA25EE26C107F5, serverId=null, dbName=_cm, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.068 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290558068000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=5597EAE1A24B2234E36E41AC54209C74, obServerId=null, obDbId=500004, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:38.068Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:38.069 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.069 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.072 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:42:38.078 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 262 2025-11-29 08:42:38.078 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.082 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290558082000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:38.082Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:42:38.084 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:37.983, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=11, sqlId=858015B92843B43D99CA77541C4C2592, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.092 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:42:38.092 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.092 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.092 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500004, sqlId=5597EAE1A24B2234E36E41AC54209C74, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.096 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.096 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.104 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.105 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=201001, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.107 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.107 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.109 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.109 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500004, sqlId=5597EAE1A24B2234E36E41AC54209C74, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.111 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:38.115 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:38.068, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=11, sqlId=5597EAE1A24B2234E36E41AC54209C74, serverId=null, dbName=_cm_hb, obDbId=500004, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.120 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.120 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=201001, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.122 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:42:38.127 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:38.082, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=11, sqlId=98E39F60591074E5DD9A6D86C7AB34D9, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.127 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:42:38.136 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=4, name=sys, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T11:20:46+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:42:38.136 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=4, name=sys, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T11:20:46+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null) 2025-11-29 08:42:38.136 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:42:38.328 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,a72ce7751e709725,c413232e38e30f5f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=a72ce7751e709725, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:38.334 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,a72ce7751e709725,c413232e38e30f5f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=a72ce7751e709725, duration=7 ms] 2025-11-29 08:42:38.341 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290558341000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1, sqlId=A5CC9FA9C8AAB2A2A93A722831E3FE6C, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=4, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:38.341Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=4, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/d 2025-11-29 08:42:38.341 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290558341000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1, sqlId=B1A1F3E7428A06C68C941CE70F77CEEE, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=4, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:38.341Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=4, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/d 2025-11-29 08:42:38.344 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290558344000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1, sqlId=D66031DC458A041BD89D3AFB4937145E, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=4, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:42:38.344Z, endTime=2025-11-29T00:40:36.591Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=4, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/d 2025-11-29 08:42:38.407 INFO 12717 --- [prometheus-minute-schedule-1,50a4f390bffba1bb,e84e6145dab689a3] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:38.414 INFO 12717 --- [prometheus-minute-schedule-1,50a4f390bffba1bb,e84e6145dab689a3] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:38.446 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 2619 2025-11-29 08:42:38.446 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.448 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 2611 2025-11-29 08:42:38.448 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.462 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.462 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1, obServerId=null, obDbId=201001, sqlId=B1A1F3E7428A06C68C941CE70F77CEEE, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.462 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.462 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1, obServerId=null, obDbId=201001, sqlId=A5CC9FA9C8AAB2A2A93A722831E3FE6C, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.466 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.466 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.471 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.471 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.482 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.482 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1, obServerId=null, obDbId=201001, sqlId=B1A1F3E7428A06C68C941CE70F77CEEE, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.485 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 10 records,use plan union hash 2025-11-29 08:42:38.487 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.487 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1, obServerId=null, obDbId=201001, sqlId=A5CC9FA9C8AAB2A2A93A722831E3FE6C, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.491 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 10 records,use plan union hash 2025-11-29 08:42:38.496 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 2611 2025-11-29 08:42:38.496 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:42:38.510 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.510 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1, obServerId=null, obDbId=201001, sqlId=D66031DC458A041BD89D3AFB4937145E, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.539 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.s.common.scheduled.OcpSchedules : Refresh OCP properties. 2025-11-29 08:42:38.539 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.ocp.core.property.PropertyService : Refresh config properties 2025-11-29 08:42:38.543 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : new: [] 2025-11-29 08:42:38.543 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : init spring 2025-11-29 08:42:38.543 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : have already run initialize. 2025-11-29 08:42:38.543 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : starting 2025-11-29 08:42:38.545 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : new: [] 2025-11-29 08:42:38.545 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : init spring 2025-11-29 08:42:38.545 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : have already run initialize. 2025-11-29 08:42:38.545 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : starting 2025-11-29 08:42:38.598 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : environmentPrepared 2025-11-29 08:42:38.602 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : contextPrepared 2025-11-29 08:42:38.602 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : set internal properties as the first PropertySource 2025-11-29 08:42:38.602 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : register bean ProgressHandler 2025-11-29 08:42:38.602 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : set internal property ocp.version to 4.3.5 2025-11-29 08:42:38.602 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : ocp.version in spring: 4.3.5 2025-11-29 08:42:38.608 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : contextLoaded 2025-11-29 08:42:38.675 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:42:38.675 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:42:38.696 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:42:38.696 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376836591000, obClusterId=1743215594, clusterName=DB, obTenantId=1, obServerId=null, obDbId=201001, sqlId=D66031DC458A041BD89D3AFB4937145E, sqlIdList=null, dynamicSql=false) 2025-11-29 08:42:38.699 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 10 records,use plan union hash 2025-11-29 08:42:38.713 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:38.341, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=4, sqlId=A5CC9FA9C8AAB2A2A93A722831E3FE6C, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.715 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:38.341, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=4, sqlId=B1A1F3E7428A06C68C941CE70F77CEEE, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.745 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:42:38.344, end=2025-11-29T08:40:36.591}, clusterId=2, tenantId=4, sqlId=D66031DC458A041BD89D3AFB4937145E, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:42:38.746 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:42:38.748 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=2, name=ocp_meta, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:42:38.748 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=2, name=ocp_meta, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null) 2025-11-29 08:42:38.748 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:42:38.750 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:42:38.751 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:38:35.085, end=2025-11-29T08:40:36.591},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:42:38.751 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=1, name=sys, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null) 2025-11-29 08:42:38.751 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:42:38.752 INFO 12717 --- [pool-18-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR, Found out 0 dubious sqls 2025-11-29 08:42:38.950 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.DataSourceInterceptor : dataSource dataSource already initialized. skipped. class: org.springframework.jdbc.datasource.DriverManagerDataSource 2025-11-29 08:42:39.334 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : ready. time taken: PT0.789030737S 2025-11-29 08:42:39.334 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : environmentPrepared 2025-11-29 08:42:39.402 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] o.s.c.c.s.e.NativeEnvironmentRepository : Adding property source: Config resource 'class path resource [config/ocp.yaml]' via location 'classpath:/config/' 2025-11-29 08:42:39.402 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] o.s.c.c.s.e.NativeEnvironmentRepository : Adding property source: Config resource 'class path resource [application.properties]' via location 'classpath:/' 2025-11-29 08:42:39.427 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] b.c.PropertySourceBootstrapConfiguration : Located property source: [BootstrapPropertySource {name='bootstrapProperties-classpath:/config/ocp.yaml'}, BootstrapPropertySource {name='bootstrapProperties-classpath:/application.properties'}, BootstrapPropertySource {name='bootstrapProperties-ocp-default'}] 2025-11-29 08:42:39.547 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : contextPrepared 2025-11-29 08:42:39.548 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : set internal properties as the first PropertySource 2025-11-29 08:42:39.548 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : register bean ProgressHandler 2025-11-29 08:42:39.548 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : set internal property ocp.version to 4.3.5 2025-11-29 08:42:39.548 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : ocp.version in spring: 4.3.5 2025-11-29 08:42:39.548 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] o.s.boot.SpringApplication : The following 1 profile is active: "composite" 2025-11-29 08:42:39.557 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : contextLoaded 2025-11-29 08:42:39.583 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:39.585 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-11-29 08:42:39.610 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:42:39.610 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:42:39.610 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:42:39.610 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:42:39.624 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:42:39.624 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:42:39.624 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:42:39.624 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:42:39.625 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:39.626 INFO 12717 --- [ocp-schedules-7,64563af69d154926,8a17f0286e367082] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-11-29 08:42:39.667 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] o.s.boot.SpringApplication : Started application in 1.124 seconds (JVM running for 2.1160369544E7) 2025-11-29 08:42:39.667 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.o.b.spring.BootstrapRunListener : ready. time taken: PT1.124816489S 2025-11-29 08:42:39.671 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:42:39.671178+08:00, lastProcessTime=2025-11-29T08:42:29.682767+08:00 2025-11-29 08:42:39.691 INFO 12717 --- [ocp-async-22,8376d340de896cb9,78c7f220375b99e7] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:39.770 INFO 12717 --- [ocp-schedules-10,b06f8e1fce72e068,ca0de1eec477247d] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for DB, save 21 to SqlAggregateData 2025-11-29 08:42:39.892 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:42:39.671178+08:00 2025-11-29 08:42:40.013 INFO 12717 --- [ocp-schedules-13,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=3 2025-11-29 08:42:40.013 INFO 12717 --- [ocp-schedules-4,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=3 2025-11-29 08:42:40.060 INFO 12717 --- [ocp-schedules-2,76420b10b4ca7b58,35cab00b6180fc22] c.o.ocp.core.property.PropertyService : Refresh config properties done 2025-11-29 08:42:40.081 INFO 12717 --- [Check obproxy3,f1b54457473da6b4,e244e37ba7158ccd] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:42:40.081 INFO 12717 --- [Check obproxy1,30a50347389f54b2,1b6c75e85ec5f4cb] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:42:40.081 INFO 12717 --- [Collect obproxy1,d7c4ae094dde5a54,b2a8f09c9f51fb39] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:42:40.081 INFO 12717 --- [Collect obproxy3,6255c0246dbe3f33,4c9f021a925c4a91] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:42:40.081 INFO 12717 --- [Check obproxy2,c3d58726358d9e56,17b30c0c238f7301] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:42:40.083 INFO 12717 --- [Collect obproxy2,3022d932c825558e,2daa47595e17ea3e] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:42:40.085 INFO 12717 --- [Check obproxy1,30a50347389f54b2,1b6c75e85ec5f4cb] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:40.088 INFO 12717 --- [Collect obproxy1,d7c4ae094dde5a54,b2a8f09c9f51fb39] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:40.089 INFO 12717 --- [Check obproxy2,c3d58726358d9e56,17b30c0c238f7301] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:40.091 INFO 12717 --- [Collect obproxy2,3022d932c825558e,2daa47595e17ea3e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:40.092 INFO 12717 --- [Collect obproxy3,6255c0246dbe3f33,4c9f021a925c4a91] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:40.094 INFO 12717 --- [Check obproxy3,f1b54457473da6b4,e244e37ba7158ccd] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:42:40.129 INFO 12717 --- [Check obproxy1,30a50347389f54b2,1b6c75e85ec5f4cb] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:42:40.131 INFO 12717 --- [Collect obproxy1,d7c4ae094dde5a54,b2a8f09c9f51fb39] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:42:40.134 INFO 12717 --- [Check obproxy2,c3d58726358d9e56,17b30c0c238f7301] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:42:40.134 INFO 12717 --- [Collect obproxy1,d7c4ae094dde5a54,b2a8f09c9f51fb39] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:42:40.134 INFO 12717 --- [Check obproxy1,30a50347389f54b2,1b6c75e85ec5f4cb] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-11-29 08:42:40.134 INFO 12717 --- [Check obproxy1,30a50347389f54b2,1b6c75e85ec5f4cb] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 1, need to sync to meta: false 2025-11-29 08:42:40.137 INFO 12717 --- [Check obproxy2,c3d58726358d9e56,17b30c0c238f7301] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-11-29 08:42:40.137 INFO 12717 --- [Check obproxy2,c3d58726358d9e56,17b30c0c238f7301] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 2, need to sync to meta: false 2025-11-29 08:42:40.139 INFO 12717 --- [Collect obproxy1,d7c4ae094dde5a54,b2a8f09c9f51fb39] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:42:40.142 INFO 12717 --- [Collect obproxy3,6255c0246dbe3f33,4c9f021a925c4a91] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:42:40.144 INFO 12717 --- [Check obproxy3,f1b54457473da6b4,e244e37ba7158ccd] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:42:40.146 INFO 12717 --- [Collect obproxy3,6255c0246dbe3f33,4c9f021a925c4a91] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:42:40.148 INFO 12717 --- [Check obproxy3,f1b54457473da6b4,e244e37ba7158ccd] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2025-11-29 08:42:40.148 INFO 12717 --- [Check obproxy3,f1b54457473da6b4,e244e37ba7158ccd] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 3 proxy_id: 3, need to sync to meta: false 2025-11-29 08:42:40.149 INFO 12717 --- [Collect obproxy3,6255c0246dbe3f33,4c9f021a925c4a91] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:42:40.149 INFO 12717 --- [Collect obproxy2,3022d932c825558e,2daa47595e17ea3e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:42:40.152 INFO 12717 --- [Collect obproxy2,3022d932c825558e,2daa47595e17ea3e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:42:40.155 INFO 12717 --- [Collect obproxy2,3022d932c825558e,2daa47595e17ea3e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:42:40.180 INFO 12717 --- [ocp-schedules-26,ff6eeab6b1236cf8,722a4675f27db22a] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:42:40.261 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,929f352d8b8e3ea2,5f68ec0728b6cb73] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=929f352d8b8e3ea2, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:40.261 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,929f352d8b8e3ea2,5f68ec0728b6cb73] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=929f352d8b8e3ea2, duration=2 ms] 2025-11-29 08:42:40.274 INFO 12717 --- [ocp-schedules-16,7c5716471e615789,8c54425f9fa8243f] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=2, successCount=2 2025-11-29 08:42:40.492 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:40.493 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:40.601 INFO 12717 --- [ocp-schedules-27,761dbf9ef1147d3e,62cb0d2084b64260] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:42:41.407 INFO 12717 --- [prometheus-minute-schedule-1,1cdb5bfc30ae93a3,f96d7446d84f75ad] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:41.414 INFO 12717 --- [prometheus-minute-schedule-1,1cdb5bfc30ae93a3,f96d7446d84f75ad] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:41.691 INFO 12717 --- [ocp-async-17,153be05b6ac67dde,a051cf943765e179] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:41.833 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-89,9ec761aef0244da2,af9a819bce61d73b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=9ec761aef0244da2, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:41.833 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-89,9ec761aef0244da2,af9a819bce61d73b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=9ec761aef0244da2, duration=1 ms] 2025-11-29 08:42:42.222 WARN 12717 --- [ocp-updater-0,,] c.o.o.c.d.service.DefaultServiceUpdater : Deleted expired service: DistributedService{name=SQL_PERF_DIAGNOSER, serverId=10.202.22.58:8080, expireInSeconds=30, createTime=2025-11-29T08:31:11+08:00, updateTime=2025-11-29T08:42:11+08:00, expireTime=null, isLeader=false} 2025-11-29 08:42:42.225 WARN 12717 --- [ocp-updater-0,,] c.o.o.c.d.service.DefaultServiceUpdater : No leader or more than one leader exists for service SQL_PERF_DIAGNOSER, Incorrect result size: expected 1, actual 0 2025-11-29 08:42:42.282 INFO 12717 --- [ocp-schedules-21,ddb9260e75292e4b,cee6f5613987fed9] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:42:42.290 INFO 12717 --- [alarm-detect-16,ddb9260e75292e4b,00945d78bc1ce030] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008306, data is empty. 2025-11-29 08:42:42.290 INFO 12717 --- [alarm-detect-2,ddb9260e75292e4b,9df674ef854cebe5] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008306, data is empty. 2025-11-29 08:42:42.292 INFO 12717 --- [alarm-detect-4,ddb9260e75292e4b,0a670cbce6a559a3] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008306, data is empty. 2025-11-29 08:42:42.295 INFO 12717 --- [alarm-detect-16,ddb9260e75292e4b,00945d78bc1ce030] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008327, data is empty. 2025-11-29 08:42:42.297 INFO 12717 --- [alarm-detect-4,ddb9260e75292e4b,0a670cbce6a559a3] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008327, data is empty. 2025-11-29 08:42:42.297 INFO 12717 --- [alarm-detect-2,ddb9260e75292e4b,9df674ef854cebe5] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008327, data is empty. 2025-11-29 08:42:42.306 WARN 12717 --- [scanner-11213811096-1,ddb9260e75292e4b,6c20f23795c4d8ee] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:42:42.361 INFO 12717 --- [alarm-detect-10,ddb9260e75292e4b,cb626b8a9d1dee9b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008306, data is empty. 2025-11-29 08:42:42.365 INFO 12717 --- [alarm-detect-10,ddb9260e75292e4b,cb626b8a9d1dee9b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008327, data is empty. 2025-11-29 08:42:42.840 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-91,767bc2510e597a4b,dc3359178678e77b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=767bc2510e597a4b, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:42.846 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-91,767bc2510e597a4b,dc3359178678e77b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=767bc2510e597a4b, duration=6 ms] 2025-11-29 08:42:43.129 INFO 12717 --- [ocp-async-8,3bd3cc4a9c814919,b6a669e9854c3a5f] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:42:43.493 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:43.495 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:43.691 INFO 12717 --- [ocp-async-32,1a0baa62df392db1,330a136a9c60f144] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:44.407 INFO 12717 --- [prometheus-minute-schedule-1,de9d750429ddc5bd,3ca686a65b46dd9f] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:44.420 INFO 12717 --- [prometheus-minute-schedule-1,de9d750429ddc5bd,3ca686a65b46dd9f] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:44.530 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-87,ada27142faabec5a,d044b64a43fe58de] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=ada27142faabec5a, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:44.534 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-87,ada27142faabec5a,d044b64a43fe58de] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=ada27142faabec5a, duration=5 ms] 2025-11-29 08:42:45.183 INFO 12717 --- [ocp-schedules-23,654656d4982e0e56,08e7df0fa610cf1a] c.o.o.s.c.scheduled.ClusterSchedules : Collect all cluster compaction info. 2025-11-29 08:42:45.185 INFO 12717 --- [ocp-schedules-23,654656d4982e0e56,08e7df0fa610cf1a] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:42:45.185 INFO 12717 --- [Collect compaction for all clusters2,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-11-29 08:42:45.185 INFO 12717 --- [Collect compaction for all clusters1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-11-29 08:42:45.691 INFO 12717 --- [ocp-async-23,cc50d8864783e24b,d82ad27229300d5f] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:46.295 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,e4dc9da8c2a0132e,badf45d9bae2cadc] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=e4dc9da8c2a0132e, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:46.295 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,e4dc9da8c2a0132e,badf45d9bae2cadc] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=e4dc9da8c2a0132e, duration=0 ms] 2025-11-29 08:42:46.495 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:46.495 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:46.964 INFO 12717 --- [ocp-schedules-9,9e2e42b7cbc00c96,21521a35e996fb27] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-11-29 08:42:46.975 INFO 12717 --- [ocp-schedules-9,9e2e42b7cbc00c96,21521a35e996fb27] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:42:46.975 INFO 12717 --- [ocp-schedules-9,9e2e42b7cbc00c96,21521a35e996fb27] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:42:47.187 INFO 12717 --- [ocp-schedules-30,5020c925feda974c,a33cd869ea99b3eb] c.o.o.s.c.scheduled.TenantSchedules : Collect all tenant major compaction 2025-11-29 08:42:47.187 INFO 12717 --- [ocp-schedules-30,5020c925feda974c,a33cd869ea99b3eb] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2025-11-29 08:42:47.238 INFO 12717 --- [Collect all tenants major compactions2,5020c925feda974c,7efa6ee0e3def816] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:ocp_monitor 2025-11-29 08:42:47.238 INFO 12717 --- [Collect all tenants major compactions1,5020c925feda974c,e57f2ce58c7b97b9] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:ocp_meta 2025-11-29 08:42:47.239 INFO 12717 --- [Collect all tenants major compactions3,5020c925feda974c,c6dafc436e95f54a] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:sys 2025-11-29 08:42:47.239 INFO 12717 --- [Collect all tenants major compactions6,5020c925feda974c,7718e077348c4dad] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:oms 2025-11-29 08:42:47.239 INFO 12717 --- [Collect all tenants major compactions4,5020c925feda974c,6d750296da09bed3] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:lhp6796 2025-11-29 08:42:47.239 INFO 12717 --- [Collect all tenants major compactions5,5020c925feda974c,7cc2adfb8ef55678] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:nextcloud 2025-11-29 08:42:47.240 INFO 12717 --- [Collect all tenants major compactions10,5020c925feda974c,ed959cf060e6424c] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:zto 2025-11-29 08:42:47.240 INFO 12717 --- [Collect all tenants major compactions7,5020c925feda974c,92abf16483afd265] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:pipeline 2025-11-29 08:42:47.250 INFO 12717 --- [Collect all tenants major compactions8,5020c925feda974c,b373b3f34e972812] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:sys 2025-11-29 08:42:47.251 INFO 12717 --- [Collect all tenants major compactions9,5020c925feda974c,2cb5e454c65af8cf] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:wecom_bot 2025-11-29 08:42:47.282 INFO 12717 --- [Collect all tenants major compactions6,5020c925feda974c,7718e077348c4dad] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:47.282 INFO 12717 --- [Collect all tenants major compactions5,5020c925feda974c,7cc2adfb8ef55678] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:47.283 INFO 12717 --- [Collect all tenants major compactions4,5020c925feda974c,6d750296da09bed3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:47.283 INFO 12717 --- [Collect all tenants major compactions6,5020c925feda974c,7718e077348c4dad] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1014] 2025-11-29 08:42:47.284 INFO 12717 --- [Collect all tenants major compactions5,5020c925feda974c,7cc2adfb8ef55678] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1018] 2025-11-29 08:42:47.285 INFO 12717 --- [Collect all tenants major compactions4,5020c925feda974c,6d750296da09bed3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1010] 2025-11-29 08:42:47.285 INFO 12717 --- [Collect all tenants major compactions1,5020c925feda974c,e57f2ce58c7b97b9] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:47.287 INFO 12717 --- [Collect all tenants major compactions1,5020c925feda974c,e57f2ce58c7b97b9] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2025-11-29 08:42:47.288 INFO 12717 --- [Collect all tenants major compactions10,5020c925feda974c,ed959cf060e6424c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:47.290 INFO 12717 --- [Collect all tenants major compactions10,5020c925feda974c,ed959cf060e6424c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1022] 2025-11-29 08:42:47.291 INFO 12717 --- [Collect all tenants major compactions2,5020c925feda974c,7efa6ee0e3def816] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:47.293 INFO 12717 --- [Collect all tenants major compactions2,5020c925feda974c,7efa6ee0e3def816] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1004] 2025-11-29 08:42:47.295 INFO 12717 --- [Collect all tenants major compactions6,5020c925feda974c,7718e077348c4dad] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:oms 2025-11-29 08:42:47.295 INFO 12717 --- [Collect all tenants major compactions9,5020c925feda974c,2cb5e454c65af8cf] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:47.295 INFO 12717 --- [Collect all tenants major compactions10,5020c925feda974c,ed959cf060e6424c] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:zto 2025-11-29 08:42:47.296 INFO 12717 --- [Collect all tenants major compactions2,5020c925feda974c,7efa6ee0e3def816] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:ocp_monitor 2025-11-29 08:42:47.296 INFO 12717 --- [Collect all tenants major compactions3,5020c925feda974c,c6dafc436e95f54a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:47.297 INFO 12717 --- [Collect all tenants major compactions9,5020c925feda974c,2cb5e454c65af8cf] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1024] 2025-11-29 08:42:47.297 INFO 12717 --- [Collect all tenants major compactions1,5020c925feda974c,e57f2ce58c7b97b9] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:ocp_meta 2025-11-29 08:42:47.298 INFO 12717 --- [Collect all tenants major compactions3,5020c925feda974c,c6dafc436e95f54a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-11-29 08:42:47.298 INFO 12717 --- [Collect all tenants major compactions8,5020c925feda974c,b373b3f34e972812] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:47.299 INFO 12717 --- [Collect all tenants major compactions5,5020c925feda974c,7cc2adfb8ef55678] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:nextcloud 2025-11-29 08:42:47.300 INFO 12717 --- [Collect all tenants major compactions3,5020c925feda974c,c6dafc436e95f54a] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:sys 2025-11-29 08:42:47.300 INFO 12717 --- [Collect all tenants major compactions8,5020c925feda974c,b373b3f34e972812] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-11-29 08:42:47.300 INFO 12717 --- [Collect all tenants major compactions7,5020c925feda974c,92abf16483afd265] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:47.302 INFO 12717 --- [Collect all tenants major compactions4,5020c925feda974c,6d750296da09bed3] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:lhp6796 2025-11-29 08:42:47.302 INFO 12717 --- [Collect all tenants major compactions7,5020c925feda974c,92abf16483afd265] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1020] 2025-11-29 08:42:47.302 INFO 12717 --- [Collect all tenants major compactions9,5020c925feda974c,2cb5e454c65af8cf] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:wecom_bot 2025-11-29 08:42:47.305 INFO 12717 --- [Collect all tenants major compactions7,5020c925feda974c,92abf16483afd265] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:pipeline 2025-11-29 08:42:47.305 INFO 12717 --- [Collect all tenants major compactions8,5020c925feda974c,b373b3f34e972812] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:sys 2025-11-29 08:42:47.306 INFO 12717 --- [ocp-schedules-30,5020c925feda974c,a33cd869ea99b3eb] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2025-11-29 08:42:47.355 INFO 12717 --- [ocp-schedules-21,c1904675e71e51d8,6b95b18108b33e61] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:42:47.358 INFO 12717 --- [ocp-schedules-21,c1904675e71e51d8,6b95b18108b33e61] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:42:47.407 INFO 12717 --- [prometheus-minute-schedule-1,81862699cf5e5528,cd279de193da89e1] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:47.415 INFO 12717 --- [prometheus-minute-schedule-1,81862699cf5e5528,cd279de193da89e1] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:47.662 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : Starting SQL_INDEX_DIAGNOSER, Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.669 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : SQL_INDEX_DIAGNOSER : for [Pair(left=Tenant(id=13, name=nextcloud, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T13:13:08+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1018, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Pair(left=Tenant(id=15, name=zto, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-06-19T19:31:51+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1022, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Pair(left=Tenant(id=16, name=wecom_bot, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-27T13:42:13+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1024, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTranspor 2025-11-29 08:42:47.682 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=13, name=nextcloud, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T13:13:08+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1018, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.691 INFO 12717 --- [ocp-async-7,78a43e3017a10e8d,2e2d6bcd9e676105] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:47.694 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=13, name=nextcloud, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T13:13:08+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1018, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.696 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=13, name=nextcloud, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T13:13:08+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1018, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.707 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=13, name=nextcloud, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T13:13:08+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1018, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.711 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=15, name=zto, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-06-19T19:31:51+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1022, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.715 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=15, name=zto, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-06-19T19:31:51+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1022, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.718 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=15, name=zto, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-06-19T19:31:51+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1022, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.722 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=15, name=zto, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-06-19T19:31:51+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1022, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.726 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=16, name=wecom_bot, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-27T13:42:13+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1024, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.729 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=16, name=wecom_bot, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-27T13:42:13+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1024, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.732 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=16, name=wecom_bot, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-27T13:42:13+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1024, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.735 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=16, name=wecom_bot, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-27T13:42:13+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1024, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.739 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=14, name=pipeline, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-29T17:10:07+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1020, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=COMPLEX_OLTP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.742 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=14, name=pipeline, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-29T17:10:07+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1020, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=COMPLEX_OLTP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.745 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=14, name=pipeline, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-29T17:10:07+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1020, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=COMPLEX_OLTP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.748 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=14, name=pipeline, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-29T17:10:07+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1020, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=COMPLEX_OLTP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.753 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=11, name=oms, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T10:08:31+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1014, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.757 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=11, name=oms, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T10:08:31+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1014, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.762 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=11, name=oms, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T10:08:31+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1014, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.767 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=11, name=oms, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T10:08:31+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1014, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.768 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,b71ba3cc8f979d88,0621990f2bf506ee] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=b71ba3cc8f979d88, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:47.768 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,b71ba3cc8f979d88,0621990f2bf506ee] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=b71ba3cc8f979d88, duration=1 ms] 2025-11-29 08:42:47.771 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.774 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.777 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.780 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.782 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=2, name=ocp_meta, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.783 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=2, name=ocp_meta, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.784 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=2, name=ocp_meta, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.785 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=2, name=ocp_meta, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.786 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=3, name=ocp_monitor, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1004, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.787 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=3, name=ocp_monitor, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1004, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.788 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=3, name=ocp_monitor, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1004, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.789 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=3, name=ocp_monitor, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1004, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.798 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=4, name=sys, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T11:20:46+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.807 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=4, name=sys, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T11:20:46+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.816 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=4, name=sys, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T11:20:46+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.823 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=4, name=sys, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T11:20:46+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.825 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.826 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.827 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.827 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:36:47.186, end=2025-11-29T08:41:47.658} 2025-11-29 08:42:47.828 INFO 12717 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : SQL_INDEX_DIAGNOSER, Found out 0 dubious sqls 2025-11-29 08:42:48.812 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,d820478e42270909,4377db8add7493b7] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=d820478e42270909, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:48.813 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,d820478e42270909,4377db8add7493b7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=d820478e42270909, duration=2 ms] 2025-11-29 08:42:48.914 INFO 12717 --- [ocp-schedules-27,59e1c56518f8d547,ea4978937a5d050f] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-11-29 08:42:49.408 INFO 12717 --- [prometheus-cache-monitor1,9b9569b480917347,1f87bf8cb9c01497] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":40357,"minuteWriteQueue":0,"minuteReadCacheSize":17215,"secondWriteCacheSize":10086,"seriesIdKeySize":17387,"secondWriteQueue":0,"minuteWriteCacheSize":17216} 2025-11-29 08:42:49.408 INFO 12717 --- [prometheus-cache-monitor1,9b9569b480917347,1f87bf8cb9c01497] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{ob_sysstat_40012=49, ob_server_resource_memory_assigned_bytes=23, ob_sysstat_40011=49, ob_sysstat_40010=49, ob_unit_num=23, ob_database_disk_required_size=74, go_memstats_mspan_sys_bytes=25, ob_sysstat_30213=42, go_memstats_stack_sys_bytes=25, ob_sysstat_30212=42, ob_sysstat_30217=42, ob_sysstat_30216=42, ob_tenant_request_queue_total_size=72, jvm_gc_memory_allocated_bytes_total=2, ob_system_event_total_waits=59, ob_tenant_disk_v4_log_disk_size=40, ob_sysstat_40019=49, ob_sysstat_40018=49, processor_process_metrics_total=30, jvm_gc_max_data_size_bytes=2, go_memstats_frees_total=25, host_available=29, ob_sysstat_30220=42, ob_sysstat_30222=42, ob_sysstat_30221=42, mysql_output_write_sql_failed_count=29, ob_database_disk_data_size=73, ob_tenant_memstore_freeze_cnt=40, ob_tenant_unit_min_memory=96, ob_sysstat_60087=49, ob_memstore_total_bytes=72, jvm_threads_states_threads=8, process_resident_memory_bytes=25, ob_cache_size_bytes=67, ob_tenant_unit_max_memory=96, input_collect_metrics_total=34, ob_sysstat_50039=50, ob_sysstat_60083=50, ob_sysstat_50038=49, node_disk_io_time_weighted_seconds_total=39, process_open_fds=25, ob_sysstat_50037=49, process_memory_rss_bytes=41, ob_server_resource_cpu_assigned_percent=23, ob_sysstat_50030=50, sql_audit_input_collect_lost_total=36, go_memstats_heap_inuse_bytes=25, ob_sysstat_40118=49, ob_sysstat_40117=49, node_memory_memfree_bytes=32, ob_sysstat_40116=49, ob_sysstat_40005=49, process_exists=50, ob_sysstat_40004=49, ob_sysstat_40003=49, ob_sysstat_40002=49, go_gc_duration_seconds=30, ob_server_resource_disk_bytes=23, ob_sysstat_40001=49, ob_sysstat_40000=49, mysql_output_write_sql_time=33, ob_sysstat_30200=42, odp_current_session=36, ob_sysstat_30201=42, ob_sysstat_30204=42, ob_sysstat_30205=42, ob_sysstat_30208=42, process_thread_count=41, ob_sysstat_30209=42, ob_sysstat_50040=50, ob_sysstat_40009=49, ob_disk_free_bytes=23, ob_sysstat_40008=49, node_filesystem_avail_bytes=67, ob_sysstat_40007=49, ob_sysstat_40006=49, node_time_seconds=31, ob_unit_config_max_i 2025-11-29 08:42:49.496 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:49.499 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:49.680 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:42:49.680042+08:00, lastProcessTime=2025-11-29T08:42:39.671178+08:00 2025-11-29 08:42:49.691 INFO 12717 --- [ocp-async-25,85112fb09d49fe52,b0be445095807d70] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:49.901 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:42:49.680042+08:00 2025-11-29 08:42:50.130 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,cdd34456baa3b493,bee1814be2233920] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=cdd34456baa3b493, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:50.130 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,cdd34456baa3b493,bee1814be2233920] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=cdd34456baa3b493, duration=1 ms] 2025-11-29 08:42:50.188 INFO 12717 --- [ocp-schedules-28,a4cb9ec191dc795b,1e251bc84e20db3f] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:42:50.407 INFO 12717 --- [prometheus-minute-schedule-1,734e5fa088e47898,c2b403fb61b521c8] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:50.413 INFO 12717 --- [prometheus-minute-schedule-1,734e5fa088e47898,c2b403fb61b521c8] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:51.692 INFO 12717 --- [ocp-async-25,b2f2c40109e4a51f,d4ed4ebf0d8668bf] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:52.386 INFO 12717 --- [ocp-schedules-14,5fd845df275a8381,0ff3f57089f932a4] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:42:52.411 WARN 12717 --- [scanner-11213811323-1,5fd845df275a8381,06dd80b511367177] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:42:52.499 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:52.500 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:53.129 INFO 12717 --- [ocp-async-31,64b699baf95d0f2f,ed7f208803045403] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:42:53.313 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,ef86f0694a075c26,d387fbd11c3c5e4a] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=ef86f0694a075c26, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:53.317 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,ef86f0694a075c26,d387fbd11c3c5e4a] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=ef86f0694a075c26, duration=4 ms] 2025-11-29 08:42:53.407 INFO 12717 --- [prometheus-minute-schedule-1,7b8175ba8f0ab0f7,7f858c91e48e673d] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:53.414 INFO 12717 --- [prometheus-minute-schedule-1,7b8175ba8f0ab0f7,7f858c91e48e673d] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:53.692 INFO 12717 --- [ocp-async-33,757281d319fce091,7d5d40a334287646] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:54.135 INFO 12717 --- [ocp-schedules-16,c1b6d2a84a64c34c,5f5f93501513771e] c.o.o.alarm.service.OcpAlarmServiceImpl : fix alarm target, alarms size 48 2025-11-29 08:42:54.695 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,d99ffa153459282e,8277fd71ddb78856] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=d99ffa153459282e, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:54.700 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,d99ffa153459282e,8277fd71ddb78856] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=d99ffa153459282e, duration=5 ms] 2025-11-29 08:42:55.086 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,323353e97b8d8ce6,5122019efdff2e63] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=323353e97b8d8ce6, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:55.086 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,323353e97b8d8ce6,5122019efdff2e63] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=323353e97b8d8ce6, duration=1 ms] 2025-11-29 08:42:55.185 INFO 12717 --- [ocp-schedules-23,654656d4982e0e56,08e7df0fa610cf1a] .o.o.s.t.b.c.AbstractShardingClusterTask : Collect compaction for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:42:55.276 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,b391e904d96f6094,8616c902866528b5] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=b391e904d96f6094, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:55.276 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,b391e904d96f6094,8616c902866528b5] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=b391e904d96f6094, duration=1 ms] 2025-11-29 08:42:55.500 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:55.502 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:55.660 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.o.c.d.lock.jdbc.DefaultLockManager : Acquired distributed lock for key part-worker 2025-11-29 08:42:55.660 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.o.s.p.PartitionRolloverService : [OCP167006] Start watching scope, scope=Monitordb 2025-11-29 08:42:55.661 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.662 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:42:55.664 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.664 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.665 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.666 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_sql_audit_stat_0 2025-11-29 08:42:55.688 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.689 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.690 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.690 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_sql_audit_stat_1 2025-11-29 08:42:55.692 INFO 12717 --- [ocp-async-26,b954f33b04ffeb73,2f6131040f6fd7a1] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:55.704 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.705 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.706 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.706 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_sql_audit_stat_2 2025-11-29 08:42:55.720 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.721 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.722 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.722 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_sql_audit_stat 2025-11-29 08:42:55.737 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.737 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.738 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.739 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_sqltext 2025-11-29 08:42:55.746 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.746 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.747 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.747 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_dynamic_sql 2025-11-29 08:42:55.753 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.754 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.755 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.755 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_high_risk_sql 2025-11-29 08:42:55.761 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.762 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.762 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.763 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_plan_cache_plan_stat_0 2025-11-29 08:42:55.769 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.770 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.771 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.771 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_plan_cache_plan 2025-11-29 08:42:55.779 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.779 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.780 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.781 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_plan_cache_plan_explain 2025-11-29 08:42:55.788 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.788 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.789 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.790 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_sql_audit_sample 2025-11-29 08:42:55.800 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.801 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.802 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.802 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_sql_audit_sample_statistics 2025-11-29 08:42:55.816 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.816 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.817 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.818 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ocp_perf_sql_diagnosis_task 2025-11-29 08:42:55.824 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.824 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.825 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.825 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ocp_perf_sql_diagnosis 2025-11-29 08:42:55.831 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.832 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.833 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.833 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_active_session_stat_0 2025-11-29 08:42:55.838 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.838 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.839 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.840 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_snapshot 2025-11-29 08:42:55.845 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.846 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.846 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.847 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_cluster 2025-11-29 08:42:55.851 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.852 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.852 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.853 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_tenant 2025-11-29 08:42:55.857 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.857 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.858 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.859 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_server 2025-11-29 08:42:55.864 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.865 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.866 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.866 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_replica 2025-11-29 08:42:55.871 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.872 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.872 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.873 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_sysstat 2025-11-29 08:42:55.877 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.878 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.879 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.879 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_row_lock 2025-11-29 08:42:55.883 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.884 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.884 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.885 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_latch 2025-11-29 08:42:55.888 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.889 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.889 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.890 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_cache 2025-11-29 08:42:55.894 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.894 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.895 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.895 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_parameter 2025-11-29 08:42:55.899 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.900 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.900 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.901 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_rpc 2025-11-29 08:42:55.905 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.906 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.906 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.907 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_packet 2025-11-29 08:42:55.911 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.911 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.912 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.912 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_clog 2025-11-29 08:42:55.916 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.917 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.917 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.918 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_location_cache 2025-11-29 08:42:55.922 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.922 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.923 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.923 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_gts 2025-11-29 08:42:55.927 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.927 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.928 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.928 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_system_event 2025-11-29 08:42:55.932 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.933 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.933 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.934 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_memory 2025-11-29 08:42:55.938 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.938 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.939 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.939 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_partition_hit 2025-11-29 08:42:55.943 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.944 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.944 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.945 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_sql_plan_monitor 2025-11-29 08:42:55.951 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.952 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.952 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.953 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_trans_stat_v2 2025-11-29 08:42:55.959 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.960 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.960 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.961 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_cluster_system_event 2025-11-29 08:42:55.966 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.966 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.967 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.967 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_cluster_server_event 2025-11-29 08:42:55.972 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.972 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.973 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.974 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ocp_metric_data_1 2025-11-29 08:42:55.977 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.977 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.978 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.978 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ocp_metric_data_60 2025-11-29 08:42:55.982 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.983 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.983 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.984 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table metric_hour_data 2025-11-29 08:42:55.987 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.988 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.988 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.989 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table metric_daily_data 2025-11-29 08:42:55.993 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.993 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:55.995 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:55.996 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table metric_data_second 2025-11-29 08:42:56.008 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.009 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:56.010 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.010 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table metric_data_minute 2025-11-29 08:42:56.035 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.036 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:56.037 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.037 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table metric_data_hour 2025-11-29 08:42:56.045 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.046 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:56.047 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.047 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table metric_data_daily 2025-11-29 08:42:56.055 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.055 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:56.056 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.057 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table metric_data_async 2025-11-29 08:42:56.064 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.065 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:56.066 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.066 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table sql_aggregate_data 2025-11-29 08:42:56.071 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.071 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:56.072 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.073 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_hist_partition_stats_compressed 2025-11-29 08:42:56.078 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.078 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:56.079 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.079 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ob_tenant_mini_compaction_info 2025-11-29 08:42:56.084 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.o.s.p.PartitionRolloverService : [OCP167007] Finish watching scope, scope=Monitordb 2025-11-29 08:42:56.084 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.o.s.p.PartitionRolloverService : [OCP167006] Start watching scope, scope=Metadb 2025-11-29 08:42:56.085 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.086 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:42:56.089 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.090 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:56.091 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.091 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table profile_credential_sharing 2025-11-29 08:42:56.099 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.099 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:42:56.101 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.101 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: show create table ocp_cache_info 2025-11-29 08:42:56.105 INFO 12717 --- [ocp-schedules-7,5ce73adc46139ff6,bddf2bd951891ca5] c.o.o.s.p.PartitionRolloverService : [OCP167007] Finish watching scope, scope=Metadb 2025-11-29 08:42:56.407 INFO 12717 --- [prometheus-minute-schedule-1,ccf727e869822648,b1718eb881c70e2b] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:56.416 INFO 12717 --- [prometheus-minute-schedule-1,ccf727e869822648,b1718eb881c70e2b] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:56.570 INFO 12717 --- [ocp-schedules-12,f361c6ff313d51b1,18907943ad2490e2] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:42:56.570 INFO 12717 --- [ocp-schedules-12,c7b1e13849e043a1,d7a296a9fce5677d] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-11-29 08:42:56.572 INFO 12717 --- [ocp-schedules-12,c7b1e13849e043a1,d7a296a9fce5677d] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:42:56.646 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.647 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:42:56.648 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.649 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:42:56.687 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:56.691 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.692 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:42:56.725 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:56.726 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.726 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:42:56.727 WARN 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=OCP 2025-11-29 08:42:56.732 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.735 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:42:56.736 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:42:56.752 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:56.752 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:56.777 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:56.787 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:56.788 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.789 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:42:56.790 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster ocpserver 2025-11-29 08:42:56.820 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2025-11-29 08:42:56.851 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:42:56.858 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:42:56.868 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:56.869 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:56.869 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:42:57.059 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:57.060 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:57.061 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:42:57.066 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:57.069 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:42:57.076 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:42:57.087 INFO 12717 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:42:57.095 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.095 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.131 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.131 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:12+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.152 INFO 12717 --- [ocp-schedules-13,7cfa59ff17f235ac,281b68af3e5417a5] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2025-11-29 08:42:57.152 INFO 12717 --- [ocp-schedules-13,7cfa59ff17f235ac,281b68af3e5417a5] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=817192494921281536 2025-11-29 08:42:57.157 INFO 12717 --- [ocp-schedules-13,7cfa59ff17f235ac,281b68af3e5417a5] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2025-11-29 08:42:57.157 INFO 12717 --- [ocp-schedules-13,7cfa59ff17f235ac,281b68af3e5417a5] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=851074473877958656 2025-11-29 08:42:57.162 INFO 12717 --- [ocp-schedules-13,7cfa59ff17f235ac,281b68af3e5417a5] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2025-11-29 08:42:57.162 INFO 12717 --- [ocp-schedules-13,7cfa59ff17f235ac,281b68af3e5417a5] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=834880106968510464 2025-11-29 08:42:57.165 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.165 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.186 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.186 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:42:57.200 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.200 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.204 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=8073, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:42:57.215 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:57.219 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:57.219 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:42:57.221 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.221 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.228 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=ocpserver, obClusterId=1743215593 2025-11-29 08:42:57.229 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=ocpserver, obClusterId=1743215593, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"ocpserver","ObClusterId":1743215593,"ObRegion":"ocpserver","ObRegionId":1743215593,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.58:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376673675000}, createTime=2025-03-29T10:43:06+08:00, creator=admin, obVersion=4.2.1.8, packageOperatingSystem=null, loadType=null, tenantCount=3) 2025-11-29 08:42:57.241 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.241 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.262 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.262 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:30+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.283 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.283 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.498 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.498 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:42:57.501 INFO 12717 --- [ocp-schedules-7,81cf6214bd1fce0a,2ef3ab27cc62cd13] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:42:57.503 INFO 12717 --- [ocp-schedules-7,81cf6214bd1fce0a,2ef3ab27cc62cd13] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:42:57.513 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.513 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.531 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.531 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:42+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.549 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.549 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:42:57.565 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:42:57.580 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:57.581 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:57.582 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:42:57.587 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster DB 2025-11-29 08:42:57.619 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 2025-11-29 08:42:57.649 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:42:57.650 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:42:57.692 INFO 12717 --- [ocp-async-35,a353c76398f86e89,f6af0f649dbdc2a5] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:42:57.874 INFO 12717 --- [Sync all clusters2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:42:58.033 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 got ObProcessInfoResult(pid=2447, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:42:58.046 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:42:58.047 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:42:58.048 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:42:58.064 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=DB, obClusterId=1743215594 2025-11-29 08:42:58.064 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=DB, obClusterId=1743215594, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=CREATED, syncStatus=VALID, rootserverJson={"ObCluster":"DB","ObClusterId":1743215594,"ObRegion":"DB","ObRegionId":1743215594,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.52:2882","role":"LEADER","sql_port":2881},{"address":"10.202.22.49:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.202.22.61:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376629943000}, createTime=2025-03-29T11:15:04+08:00, creator=admin, obVersion=4.3.5.1, packageOperatingSystem=el8, loadType=HTAP, tenantCount=7) 2025-11-29 08:42:58.427 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,d6c104a3cb74f95f,a2e2f9e5ca1ff4bd] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=d6c104a3cb74f95f, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:42:58.427 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,d6c104a3cb74f95f,a2e2f9e5ca1ff4bd] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=d6c104a3cb74f95f, duration=1 ms] 2025-11-29 08:42:58.502 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:42:58.503 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:42:58.976 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,47ca16ce02112c07,a97decffb5a28f06] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=47ca16ce02112c07, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:42:58.977 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,47ca16ce02112c07,a97decffb5a28f06] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=47ca16ce02112c07, duration=2 ms] 2025-11-29 08:42:59.407 INFO 12717 --- [prometheus-minute-schedule-1,04bf9141e1337fbd,6b3a5684e2aaaee8] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764376980, interval=60 2025-11-29 08:42:59.418 INFO 12717 --- [prometheus-minute-schedule-1,04bf9141e1337fbd,6b3a5684e2aaaee8] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:42:59.680 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:42:59.679630+08:00, lastProcessTime=2025-11-29T08:42:49.680042+08:00 2025-11-29 08:42:59.692 INFO 12717 --- [ocp-async-35,520371a741331f11,2dc246d2e2a2d231] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:00.030 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:42:59.679630+08:00 2025-11-29 08:43:00.193 INFO 12717 --- [ocp-schedules-29,56d84c05d1a3ce76,d3b6f70de74758ab] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:43:00.266 INFO 12717 --- [ocp-schedules-23,1aad69a8805b57a3,74f126ff3beabdba] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=BackupAlarmCheckSchedule, method=run 2025-11-29 08:43:00.270 INFO 12717 --- [ocp-schedules-20,794693a31650aae5,79f378fc42fca149] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2025-11-29 08:43:00.271 INFO 12717 --- [ocp-schedules-30,b6c78f84d561a01a,e8fec9ceb3ebc9f4] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-11-29 08:43:00.271 INFO 12717 --- [ocp-schedules-7,b2cb7d4b9baa73fb,e012008c645547c6] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-11-29 08:43:00.271 INFO 12717 --- [ocp-schedules-1,95d64e6e9f195606,267c6896e4021eac] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-11-29 08:43:00.276 INFO 12717 --- [ocp-schedules-7,b2cb7d4b9baa73fb,e012008c645547c6] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:43:00.277 INFO 12717 --- [ocp-schedules-1,95d64e6e9f195606,267c6896e4021eac] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:43:00.297 INFO 12717 --- [ocp-schedules-20,794693a31650aae5,79f378fc42fca149] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2025-11-29 08:43:00.315 INFO 12717 --- [ocp-schedules-8,870347e56eb4e822,009158576dbb30b2] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:43:00.316 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:43:00.316 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:43:00.316 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:43:00.317 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:43:00.317 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:43:00.317 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:43:00.317 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:43:00.318 INFO 12717 --- [ocp-schedules-2,9ab4fd119acfdda1,271aabd99a97e56d] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:43:00.318 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:43:00.318 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:43:00.318 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:43:00.318 INFO 12717 --- [ocp-schedules-28,d7cd6a2fce7129aa,78a25175aa05d486] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:43:00.318 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:43:00.319 INFO 12717 --- [host-check-clockdiff1,d7cd6a2fce7129aa,b0411d8167f74402] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.58], timeoutMillis=20000 2025-11-29 08:43:00.319 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:43:00.319 INFO 12717 --- [host-check-agent-status2,9ab4fd119acfdda1,b855c8c2de6a0dc8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:43:00.320 INFO 12717 --- [host-check-agent-status3,9ab4fd119acfdda1,5994ddcf383e76fc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:43:00.320 INFO 12717 --- [host-check-agent-status1,9ab4fd119acfdda1,745c15f100c1387f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:43:00.320 INFO 12717 --- [host-check-clockdiff2,d7cd6a2fce7129aa,2cbc4e5401add9ad] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.50], timeoutMillis=20000 2025-11-29 08:43:00.320 INFO 12717 --- [host-check-clockdiff5,d7cd6a2fce7129aa,5820b7087a7eb47f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.53], timeoutMillis=20000 2025-11-29 08:43:00.320 INFO 12717 --- [host-check-agent-status5,9ab4fd119acfdda1,8509afa7df6ef24e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:43:00.322 INFO 12717 --- [host-check-clockdiff12,d7cd6a2fce7129aa,58d2f0552724cf5d] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.49], timeoutMillis=20000 2025-11-29 08:43:00.322 INFO 12717 --- [host-check-agent-status7,9ab4fd119acfdda1,78cde63abf82abd3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:43:00.323 INFO 12717 --- [host-check-agent-status6,9ab4fd119acfdda1,d5c5ecb3d2bb846b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:43:00.323 INFO 12717 --- [host-check-agent-status4,9ab4fd119acfdda1,668d26682a850709] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:43:00.323 INFO 12717 --- [host-check-agent-status8,9ab4fd119acfdda1,68d20e8dc18a7e1f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:43:00.323 INFO 12717 --- [host-check-agent-status9,9ab4fd119acfdda1,8c6d512481c36980] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:43:00.323 INFO 12717 --- [host-check-clockdiff10,d7cd6a2fce7129aa,75efd51ce8411132] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.47], timeoutMillis=20000 2025-11-29 08:43:00.323 INFO 12717 --- [host-check-clockdiff3,d7cd6a2fce7129aa,d905e7e770600fdf] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.51], timeoutMillis=20000 2025-11-29 08:43:00.323 INFO 12717 --- [host-check-clockdiff7,d7cd6a2fce7129aa,57c877280b8db9a4] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.59], timeoutMillis=20000 2025-11-29 08:43:00.323 INFO 12717 --- [host-check-clockdiff8,d7cd6a2fce7129aa,ab9d606f5bab54a9] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.60], timeoutMillis=20000 2025-11-29 08:43:00.323 INFO 12717 --- [host-check-clockdiff6,d7cd6a2fce7129aa,04eb6eb9c655b8c3] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.54], timeoutMillis=20000 2025-11-29 08:43:00.323 INFO 12717 --- [host-check-clockdiff9,d7cd6a2fce7129aa,0a9bf0d50b2416a7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.61], timeoutMillis=20000 2025-11-29 08:43:00.326 INFO 12717 --- [host-check-clockdiff4,d7cd6a2fce7129aa,4ba5ef832c0045e8] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.52], timeoutMillis=20000 2025-11-29 08:43:00.326 INFO 12717 --- [host-check-clockdiff11,d7cd6a2fce7129aa,1872b532ee7cdad4] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.48], timeoutMillis=20000 2025-11-29 08:43:00.327 INFO 12717 --- [host-check-agent-status10,9ab4fd119acfdda1,e60d39508684e6b2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:43:00.329 INFO 12717 --- [host-check-agent-status12,9ab4fd119acfdda1,669f403a398361cf] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:43:00.329 INFO 12717 --- [host-check-agent-status11,9ab4fd119acfdda1,d55647b7ecc4de23] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:43:00.341 INFO 12717 --- [host-check-clockdiff1,d7cd6a2fce7129aa,b0411d8167f74402] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376980 0 0 , error= 2025-11-29 08:43:00.363 INFO 12717 --- [ocp-schedules-23,1aad69a8805b57a3,74f126ff3beabdba] c.o.o.b.i.alarm.BackupAlarmService : Backup alarm check schedule. 2025-11-29 08:43:00.369 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:43:00.370 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:43:00.370 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:43:00.370 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:43:00.370 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:43:00.371 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:43:00.371 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:43:00.372 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:43:00.372 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:43:00.372 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:43:00.378 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:43:00.379 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:43:00.380 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:43:00.380 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:43:00.380 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:43:00.381 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:43:00.381 INFO 12717 --- [ocp-schedules-17,3ea2c92e4ee08349,adcbc6821423c4bf] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-11-29 08:43:00.380 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:43:00.381 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:43:00.381 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:43:00.381 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:43:00.382 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:43:00.384 INFO 12717 --- [host-check-agent-status1,9ab4fd119acfdda1,745c15f100c1387f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:43:00.384 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:43:00.384 INFO 12717 --- [host-check-agent-status5,9ab4fd119acfdda1,8509afa7df6ef24e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:43:00.385 INFO 12717 --- [host-check-agent-status10,9ab4fd119acfdda1,e60d39508684e6b2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:43:00.385 INFO 12717 --- [host-check-agent-status3,9ab4fd119acfdda1,5994ddcf383e76fc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:43:00.385 INFO 12717 --- [host-check-agent-status9,9ab4fd119acfdda1,8c6d512481c36980] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:43:00.385 INFO 12717 --- [host-check-agent-status4,9ab4fd119acfdda1,668d26682a850709] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:43:00.386 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:43:00.386 INFO 12717 --- [host-check-agent-status7,9ab4fd119acfdda1,78cde63abf82abd3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:43:00.386 INFO 12717 --- [host-check-agent-status6,9ab4fd119acfdda1,d5c5ecb3d2bb846b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:43:00.387 INFO 12717 --- [host-check-agent-status2,9ab4fd119acfdda1,b855c8c2de6a0dc8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:43:00.387 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:43:00.387 INFO 12717 --- [host-check-agent-status1,9ab4fd119acfdda1,745c15f100c1387f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:43:00.388 INFO 12717 --- [host-check-agent-status11,9ab4fd119acfdda1,d55647b7ecc4de23] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:43:00.388 INFO 12717 --- [host-check-agent-status12,9ab4fd119acfdda1,669f403a398361cf] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:43:00.389 INFO 12717 --- [host-check-agent-status10,9ab4fd119acfdda1,e60d39508684e6b2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:43:00.389 INFO 12717 --- [host-check-agent-status3,9ab4fd119acfdda1,5994ddcf383e76fc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:43:00.389 INFO 12717 --- [host-check-agent-status5,9ab4fd119acfdda1,8509afa7df6ef24e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:43:00.389 INFO 12717 --- [host-check-agent-status6,9ab4fd119acfdda1,d5c5ecb3d2bb846b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:43:00.389 INFO 12717 --- [host-check-agent-status8,9ab4fd119acfdda1,68d20e8dc18a7e1f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:43:00.389 INFO 12717 --- [host-check-agent-status4,9ab4fd119acfdda1,668d26682a850709] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:43:00.390 INFO 12717 --- [host-check-agent-status9,9ab4fd119acfdda1,8c6d512481c36980] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:43:00.390 INFO 12717 --- [host-check-agent-status2,9ab4fd119acfdda1,b855c8c2de6a0dc8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:43:00.390 INFO 12717 --- [host-check-agent-status7,9ab4fd119acfdda1,78cde63abf82abd3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:43:00.394 INFO 12717 --- [host-check-agent-status12,9ab4fd119acfdda1,669f403a398361cf] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:43:00.394 INFO 12717 --- [host-check-agent-status11,9ab4fd119acfdda1,d55647b7ecc4de23] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:43:00.395 INFO 12717 --- [host-check-agent-status8,9ab4fd119acfdda1,68d20e8dc18a7e1f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:43:00.504 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:00.505 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:43:00.506 INFO 12717 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.8 2025-11-29 08:43:00.507 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:00.508 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:43:00.518 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/time, params:null 2025-11-29 08:43:00.518 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/time, params:null 2025-11-29 08:43:00.519 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/time, params:null 2025-11-29 08:43:00.519 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/time, params:null 2025-11-29 08:43:00.521 INFO 12717 --- [host-check-agent-status1,9ab4fd119acfdda1,745c15f100c1387f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.521 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/time, params:null 2025-11-29 08:43:00.521 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.202.22.51, timestamp=2025-11-29T08:43:00.518224278+08:00 2025-11-29 08:43:00.521 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.202.22.53, timestamp=2025-11-29T08:43:00.520658557+08:00 2025-11-29 08:43:00.521 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/time, params:null 2025-11-29 08:43:00.522 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/time, params:null 2025-11-29 08:43:00.522 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-11-29T08:43:00.521+08:00 2025-11-29 08:43:00.522 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/time, params:null 2025-11-29 08:43:00.522 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-11-29T08:43:00.521+08:00 2025-11-29 08:43:00.523 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.202.22.54, timestamp=2025-11-29T08:43:00.518797254+08:00 2025-11-29 08:43:00.523 INFO 12717 --- [host-check-agent-status10,9ab4fd119acfdda1,e60d39508684e6b2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.523 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/time, params:null 2025-11-29 08:43:00.523 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.202.22.52, timestamp=2025-11-29T08:43:00.521244117+08:00 2025-11-29 08:43:00.523 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/time, params:null 2025-11-29 08:43:00.524 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2025-11-29T08:43:00.523+08:00 2025-11-29 08:43:00.524 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=11, ip=10.202.22.47, timestamp=2025-11-29T08:43:00.523156697+08:00 2025-11-29 08:43:00.524 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.202.22.50, timestamp=2025-11-29T08:43:00.524139073+08:00 2025-11-29 08:43:00.525 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=13, ip=10.202.22.49, timestamp=2025-11-29T08:43:00.523743461+08:00 2025-11-29 08:43:00.525 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-11-29T08:43:00.523+08:00 2025-11-29 08:43:00.525 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=11, lastAvailableTime=2025-11-29T08:43:00.524+08:00 2025-11-29 08:43:00.526 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/time, params:null 2025-11-29 08:43:00.526 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=13, lastAvailableTime=2025-11-29T08:43:00.525+08:00 2025-11-29 08:43:00.526 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-11-29T08:43:00.524+08:00 2025-11-29 08:43:00.527 INFO 12717 --- [host-check-agent-status8,9ab4fd119acfdda1,68d20e8dc18a7e1f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.527 INFO 12717 --- [host-check-agent-status1,9ab4fd119acfdda1,745c15f100c1387f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.58 2025-11-29 08:43:00.527 INFO 12717 --- [host-check-agent-status1,9ab4fd119acfdda1,745c15f100c1387f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.58 2025-11-29 08:43:00.527 INFO 12717 --- [host-check-agent-status11,9ab4fd119acfdda1,d55647b7ecc4de23] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.527 INFO 12717 --- [host-check-agent-status1,9ab4fd119acfdda1,745c15f100c1387f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.58 2025-11-29 08:43:00.528 INFO 12717 --- [host-check-agent-status6,9ab4fd119acfdda1,d5c5ecb3d2bb846b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.529 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/time, params:null 2025-11-29 08:43:00.529 INFO 12717 --- [host-check-agent-status4,9ab4fd119acfdda1,668d26682a850709] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.530 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-11-29T08:43:00.521+08:00 and status AVAILABLE 2025-11-29 08:43:00.530 INFO 12717 --- [host-check-agent-status10,9ab4fd119acfdda1,e60d39508684e6b2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.47 2025-11-29 08:43:00.530 INFO 12717 --- [host-check-agent-status10,9ab4fd119acfdda1,e60d39508684e6b2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.47 2025-11-29 08:43:00.530 INFO 12717 --- [host-check-agent-status10,9ab4fd119acfdda1,e60d39508684e6b2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.47 2025-11-29 08:43:00.530 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.202.22.58, timestamp=2025-11-29T08:43:00.530011882+08:00 2025-11-29 08:43:00.530 INFO 12717 --- [host-check-agent-status2,9ab4fd119acfdda1,b855c8c2de6a0dc8] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.531 INFO 12717 --- [host-check-agent-status3,9ab4fd119acfdda1,5994ddcf383e76fc] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.531 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-11-29T08:43:00.530+08:00 2025-11-29 08:43:00.532 INFO 12717 --- [host-check-agent-status12,9ab4fd119acfdda1,669f403a398361cf] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.532 INFO 12717 --- [host-check-agent-status6,9ab4fd119acfdda1,d5c5ecb3d2bb846b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.54 2025-11-29 08:43:00.532 INFO 12717 --- [host-check-agent-status6,9ab4fd119acfdda1,d5c5ecb3d2bb846b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.54 2025-11-29 08:43:00.532 INFO 12717 --- [host-check-agent-status6,9ab4fd119acfdda1,d5c5ecb3d2bb846b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.54 2025-11-29 08:43:00.532 INFO 12717 --- [host-check-agent-status7,9ab4fd119acfdda1,78cde63abf82abd3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.532 INFO 12717 --- [host-check-agent-status4,9ab4fd119acfdda1,668d26682a850709] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.52 2025-11-29 08:43:00.532 INFO 12717 --- [host-check-agent-status4,9ab4fd119acfdda1,668d26682a850709] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.52 2025-11-29 08:43:00.532 INFO 12717 --- [host-check-agent-status4,9ab4fd119acfdda1,668d26682a850709] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.52 2025-11-29 08:43:00.532 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-11-29T08:43:00.523+08:00 and status AVAILABLE 2025-11-29 08:43:00.533 INFO 12717 --- [host-check-agent-status9,9ab4fd119acfdda1,8c6d512481c36980] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.533 INFO 12717 --- [host-check-agent-status5,9ab4fd119acfdda1,8509afa7df6ef24e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/agent/status, params:null 2025-11-29 08:43:00.534 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-11-29T08:43:00.530+08:00 and status AVAILABLE 2025-11-29 08:43:00.534 INFO 12717 --- [host-check-agent-status2,9ab4fd119acfdda1,b855c8c2de6a0dc8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.50 2025-11-29 08:43:00.534 INFO 12717 --- [host-check-agent-status2,9ab4fd119acfdda1,b855c8c2de6a0dc8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.50 2025-11-29 08:43:00.534 INFO 12717 --- [host-check-agent-status2,9ab4fd119acfdda1,b855c8c2de6a0dc8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.50 2025-11-29 08:43:00.534 INFO 12717 --- [host-check-agent-status3,9ab4fd119acfdda1,5994ddcf383e76fc] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.51 2025-11-29 08:43:00.534 INFO 12717 --- [host-check-agent-status3,9ab4fd119acfdda1,5994ddcf383e76fc] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.51 2025-11-29 08:43:00.534 INFO 12717 --- [host-check-agent-status3,9ab4fd119acfdda1,5994ddcf383e76fc] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.51 2025-11-29 08:43:00.535 INFO 12717 --- [host-check-agent-status12,9ab4fd119acfdda1,669f403a398361cf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.49 2025-11-29 08:43:00.535 INFO 12717 --- [host-check-agent-status12,9ab4fd119acfdda1,669f403a398361cf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.49 2025-11-29 08:43:00.535 INFO 12717 --- [host-check-agent-status12,9ab4fd119acfdda1,669f403a398361cf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.49 2025-11-29 08:43:00.536 INFO 12717 --- [host-check-agent-status5,9ab4fd119acfdda1,8509afa7df6ef24e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.53 2025-11-29 08:43:00.536 INFO 12717 --- [host-check-agent-status5,9ab4fd119acfdda1,8509afa7df6ef24e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.53 2025-11-29 08:43:00.536 INFO 12717 --- [host-check-agent-status5,9ab4fd119acfdda1,8509afa7df6ef24e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.53 2025-11-29 08:43:00.537 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.s.compute.HostAgentDaoManager : Updated agent 11 last available time 2025-11-29T08:43:00.524+08:00 and status AVAILABLE 2025-11-29 08:43:00.540 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=8, ip=10.202.22.59, timestamp=2025-11-29T08:43:00.538304755+08:00 2025-11-29 08:43:00.541 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=8, lastAvailableTime=2025-11-29T08:43:00.540+08:00 2025-11-29 08:43:00.541 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-11-29T08:43:00.521+08:00 and status AVAILABLE 2025-11-29 08:43:00.543 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2025-11-29T08:43:00.524+08:00 and status AVAILABLE 2025-11-29 08:43:00.543 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-11-29T08:43:00.523+08:00 and status AVAILABLE 2025-11-29 08:43:00.544 INFO 12717 --- [host-check-agent-status7,9ab4fd119acfdda1,78cde63abf82abd3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.59 2025-11-29 08:43:00.544 INFO 12717 --- [host-check-agent-status7,9ab4fd119acfdda1,78cde63abf82abd3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.59 2025-11-29 08:43:00.544 INFO 12717 --- [host-check-agent-status7,9ab4fd119acfdda1,78cde63abf82abd3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.59 2025-11-29 08:43:00.544 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.s.compute.HostAgentDaoManager : Updated agent 8 last available time 2025-11-29T08:43:00.540+08:00 and status AVAILABLE 2025-11-29 08:43:00.545 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.s.compute.HostAgentDaoManager : Updated agent 12 last available time 2025-11-29T08:43:00.525+08:00 and status AVAILABLE 2025-11-29 08:43:00.565 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=12, ip=10.202.22.48, timestamp=2025-11-29T08:43:00.562453779+08:00 2025-11-29 08:43:00.566 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=12, lastAvailableTime=2025-11-29T08:43:00.565+08:00 2025-11-29 08:43:00.570 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.s.compute.HostAgentDaoManager : Updated agent 13 last available time 2025-11-29T08:43:00.565+08:00 and status AVAILABLE 2025-11-29 08:43:00.658 INFO 12717 --- [host-check-agent-status11,9ab4fd119acfdda1,d55647b7ecc4de23] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.48 2025-11-29 08:43:00.658 INFO 12717 --- [host-check-agent-status11,9ab4fd119acfdda1,d55647b7ecc4de23] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.48 2025-11-29 08:43:00.658 INFO 12717 --- [host-check-agent-status11,9ab4fd119acfdda1,d55647b7ecc4de23] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.48 2025-11-29 08:43:00.714 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=9, ip=10.202.22.60, timestamp=2025-11-29T08:43:00.708440649+08:00 2025-11-29 08:43:00.714 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=10, ip=10.202.22.61, timestamp=2025-11-29T08:43:00.712452376+08:00 2025-11-29 08:43:00.715 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=10, lastAvailableTime=2025-11-29T08:43:00.714+08:00 2025-11-29 08:43:00.715 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=9, lastAvailableTime=2025-11-29T08:43:00.714+08:00 2025-11-29 08:43:00.718 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2025-11-29T08:43:00.714+08:00 and status AVAILABLE 2025-11-29 08:43:00.718 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.s.compute.HostAgentDaoManager : Updated agent 10 last available time 2025-11-29T08:43:00.714+08:00 and status AVAILABLE 2025-11-29 08:43:00.724 INFO 12717 --- [host-check-agent-status9,9ab4fd119acfdda1,8c6d512481c36980] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.61 2025-11-29 08:43:00.724 INFO 12717 --- [host-check-agent-status9,9ab4fd119acfdda1,8c6d512481c36980] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.61 2025-11-29 08:43:00.724 INFO 12717 --- [host-check-agent-status9,9ab4fd119acfdda1,8c6d512481c36980] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.61 2025-11-29 08:43:00.728 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-11-29 08:43:00.728 INFO 12717 --- [Check status for all clusters2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.3.5.1 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=13 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=11 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=10 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=12 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-agent-status8,9ab4fd119acfdda1,68d20e8dc18a7e1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.60 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-agent-status8,9ab4fd119acfdda1,68d20e8dc18a7e1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.60 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-agent-status8,9ab4fd119acfdda1,68d20e8dc18a7e1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.60 2025-11-29 08:43:00.729 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=9 2025-11-29 08:43:00.730 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=8 2025-11-29 08:43:00.730 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-11-29 08:43:00.731 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-11-29 08:43:00.765 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:43:00.765 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:43:00.765 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:43:00.766 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:43:00.766 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:43:00.766 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:43:00.766 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:43:00.766 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:43:00.766 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:43:00.767 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:43:00.767 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:43:00.767 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:43:00.767 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:43:00.767 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:43:00.767 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:43:00.767 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:43:00.767 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:43:00.767 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:43:00.768 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:43:00.768 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:43:00.768 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:43:00.769 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:43:00.770 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:43:00.772 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:43:00.797 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.798 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.799 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.799 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.799 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.800 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.800 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.800 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.802 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.802 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.802 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.805 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:00.850 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=31 2025-11-29 08:43:00.854 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-11-29 08:43:00.856 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-11-29 08:43:00.857 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=16 2025-11-29 08:43:00.859 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-11-29 08:43:00.860 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=25 2025-11-29 08:43:00.863 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=28 2025-11-29 08:43:00.863 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=34 2025-11-29 08:43:00.864 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-11-29 08:43:00.868 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-11-29 08:43:00.877 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-11-29 08:43:00.877 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=31 2025-11-29 08:43:00.877 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=25 2025-11-29 08:43:00.878 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=28 2025-11-29 08:43:00.878 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-11-29 08:43:00.878 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=16 2025-11-29 08:43:00.878 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=34 2025-11-29 08:43:00.879 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-11-29 08:43:00.880 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-11-29 08:43:00.885 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-11-29 08:43:01.040 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=22 2025-11-29 08:43:01.048 WARN 12717 --- [host-check-clockdiff10,d7cd6a2fce7129aa,75efd51ce8411132] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.47], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:01.048 ERROR 12717 --- [host-check-clockdiff10,d7cd6a2fce7129aa,75efd51ce8411132] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.47] 2025-11-29 08:43:01.049 WARN 12717 --- [host-check-clockdiff10,d7cd6a2fce7129aa,75efd51ce8411132] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=11, ip=10.202.22.47, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.47 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:01.050 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=22 2025-11-29 08:43:01.085 INFO 12717 --- [ocp-schedules-5,97968794d175bc2a,29f363c0390eb110] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration auto replace status 2025-11-29 08:43:01.097 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.097 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.098 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.098 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.098 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.098 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.098 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.099 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.100 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.101 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.102 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.147 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-11-29 08:43:01.148 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=17 2025-11-29 08:43:01.151 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-11-29 08:43:01.158 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-11-29 08:43:01.162 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=35 2025-11-29 08:43:01.162 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=17 2025-11-29 08:43:01.164 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-11-29 08:43:01.164 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-11-29 08:43:01.166 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-11-29 08:43:01.168 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-11-29 08:43:01.172 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-11-29 08:43:01.173 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=32 2025-11-29 08:43:01.176 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=35 2025-11-29 08:43:01.180 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-11-29 08:43:01.180 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-11-29 08:43:01.185 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=32 2025-11-29 08:43:01.188 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=26 2025-11-29 08:43:01.190 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.192 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.196 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.198 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.198 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.198 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=26 2025-11-29 08:43:01.201 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=29 2025-11-29 08:43:01.202 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.205 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.208 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.212 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=29 2025-11-29 08:43:01.219 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.228 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=37 2025-11-29 08:43:01.233 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.234 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-11-29 08:43:01.239 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=37 2025-11-29 08:43:01.242 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=18 2025-11-29 08:43:01.245 INFO 12717 --- [host-check-host-status4,870347e56eb4e822,c7df5a6e49978333] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-11-29 08:43:01.250 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-11-29 08:43:01.253 INFO 12717 --- [host-check-host-status2,870347e56eb4e822,bc482c97f235b4b5] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=18 2025-11-29 08:43:01.260 INFO 12717 --- [host-check-host-status6,870347e56eb4e822,59a05756b655db10] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-11-29 08:43:01.262 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-11-29 08:43:01.262 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=33 2025-11-29 08:43:01.263 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=36 2025-11-29 08:43:01.264 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-11-29 08:43:01.272 INFO 12717 --- [host-check-host-status12,870347e56eb4e822,4ba3d4ada809c605] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=36 2025-11-29 08:43:01.273 INFO 12717 --- [host-check-host-status3,870347e56eb4e822,b3a1218b68f5997a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-11-29 08:43:01.273 INFO 12717 --- [host-check-host-status10,870347e56eb4e822,ae9cce9500997d30] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=33 2025-11-29 08:43:01.274 INFO 12717 --- [host-check-host-status5,870347e56eb4e822,675d9a0c4fe718eb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-11-29 08:43:01.278 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-11-29 08:43:01.286 INFO 12717 --- [host-check-host-status1,870347e56eb4e822,4c9b2de896359902] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-11-29 08:43:01.298 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=23 2025-11-29 08:43:01.309 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=23 2025-11-29 08:43:01.453 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:01.458 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.490 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=27 2025-11-29 08:43:01.500 INFO 12717 --- [host-check-host-status9,870347e56eb4e822,1dbb2b7b7d34a3f3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=27 2025-11-29 08:43:01.503 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:01.504 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:01.522 WARN 12717 --- [host-check-clockdiff4,d7cd6a2fce7129aa,4ba5ef832c0045e8] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.52], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:01.522 ERROR 12717 --- [host-check-clockdiff4,d7cd6a2fce7129aa,4ba5ef832c0045e8] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.52] 2025-11-29 08:43:01.522 WARN 12717 --- [host-check-clockdiff4,d7cd6a2fce7129aa,4ba5ef832c0045e8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.202.22.52, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.52 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:01.522 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=30 2025-11-29 08:43:01.530 INFO 12717 --- [host-check-host-status8,870347e56eb4e822,eadcb56f753244c1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=30 2025-11-29 08:43:01.539 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=38 2025-11-29 08:43:01.548 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=38 2025-11-29 08:43:01.564 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:01.674 WARN 12717 --- [host-check-clockdiff7,d7cd6a2fce7129aa,57c877280b8db9a4] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.59], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:01.674 ERROR 12717 --- [host-check-clockdiff7,d7cd6a2fce7129aa,57c877280b8db9a4] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.59] 2025-11-29 08:43:01.674 WARN 12717 --- [host-check-clockdiff7,d7cd6a2fce7129aa,57c877280b8db9a4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=8, ip=10.202.22.59, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.59 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:01.677 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=24 2025-11-29 08:43:01.686 INFO 12717 --- [host-check-host-status7,870347e56eb4e822,bf81822ea574c075] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=24 2025-11-29 08:43:01.692 INFO 12717 --- [ocp-async-21,5b61fbc2473758be,74aa9fec3b51d98f] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:01.732 WARN 12717 --- [host-check-clockdiff2,d7cd6a2fce7129aa,2cbc4e5401add9ad] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.50], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:01.732 ERROR 12717 --- [host-check-clockdiff2,d7cd6a2fce7129aa,2cbc4e5401add9ad] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.50] 2025-11-29 08:43:01.732 WARN 12717 --- [host-check-clockdiff2,d7cd6a2fce7129aa,2cbc4e5401add9ad] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.202.22.50, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.50 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:01.844 WARN 12717 --- [host-check-clockdiff6,d7cd6a2fce7129aa,04eb6eb9c655b8c3] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.54], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:01.844 ERROR 12717 --- [host-check-clockdiff6,d7cd6a2fce7129aa,04eb6eb9c655b8c3] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.54] 2025-11-29 08:43:01.844 WARN 12717 --- [host-check-clockdiff6,d7cd6a2fce7129aa,04eb6eb9c655b8c3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.202.22.54, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.54 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:01.854 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=39 2025-11-29 08:43:01.864 INFO 12717 --- [host-check-host-status11,870347e56eb4e822,ec5327e12ed21835] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=39 2025-11-29 08:43:02.021 INFO 12717 --- [host-check-clockdiff3,d7cd6a2fce7129aa,d905e7e770600fdf] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376982 -3 -3 , error= 2025-11-29 08:43:02.212 WARN 12717 --- [host-check-clockdiff5,d7cd6a2fce7129aa,5820b7087a7eb47f] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.53], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:02.213 ERROR 12717 --- [host-check-clockdiff5,d7cd6a2fce7129aa,5820b7087a7eb47f] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.53] 2025-11-29 08:43:02.213 WARN 12717 --- [host-check-clockdiff5,d7cd6a2fce7129aa,5820b7087a7eb47f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.202.22.53, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.53 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:02.399 WARN 12717 --- [host-check-clockdiff12,d7cd6a2fce7129aa,58d2f0552724cf5d] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.49], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:02.399 ERROR 12717 --- [host-check-clockdiff12,d7cd6a2fce7129aa,58d2f0552724cf5d] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.49] 2025-11-29 08:43:02.399 WARN 12717 --- [host-check-clockdiff12,d7cd6a2fce7129aa,58d2f0552724cf5d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=13, ip=10.202.22.49, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.49 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:02.407 INFO 12717 --- [prometheus-minute-schedule-1,289bacdc75582161,1a12e48847f2de5f] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:02.414 INFO 12717 --- [prometheus-minute-schedule-1,289bacdc75582161,1a12e48847f2de5f] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:02.482 INFO 12717 --- [ocp-schedules-29,e2e2bf5ee919cdf1,c73b9565e7172f6f] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:43:02.488 INFO 12717 --- [alarm-detect-15,e2e2bf5ee919cdf1,1cc83798cbdbd088] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008327, data is empty. 2025-11-29 08:43:02.488 INFO 12717 --- [alarm-detect-17,e2e2bf5ee919cdf1,9aca2b696a0bac42] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008327, data is empty. 2025-11-29 08:43:02.489 INFO 12717 --- [alarm-detect-4,e2e2bf5ee919cdf1,38c0798912726315] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008327, data is empty. 2025-11-29 08:43:02.493 INFO 12717 --- [alarm-detect-17,e2e2bf5ee919cdf1,9aca2b696a0bac42] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008327, data is empty. 2025-11-29 08:43:02.494 INFO 12717 --- [alarm-detect-15,e2e2bf5ee919cdf1,1cc83798cbdbd088] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008327, data is empty. 2025-11-29 08:43:02.494 INFO 12717 --- [alarm-detect-4,e2e2bf5ee919cdf1,38c0798912726315] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008327, data is empty. 2025-11-29 08:43:02.502 WARN 12717 --- [scanner-11213811516-1,e2e2bf5ee919cdf1,ed58eb1916cc804b] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:43:02.570 INFO 12717 --- [alarm-detect-18,e2e2bf5ee919cdf1,cdcd376a06db426f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008327, data is empty. 2025-11-29 08:43:02.577 INFO 12717 --- [alarm-detect-18,e2e2bf5ee919cdf1,cdcd376a06db426f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008327, data is empty. 2025-11-29 08:43:02.784 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,c670a1b94bdb8707,5a337157c97b358e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=c670a1b94bdb8707, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:02.784 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,c670a1b94bdb8707,5a337157c97b358e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=c670a1b94bdb8707, duration=1 ms] 2025-11-29 08:43:03.129 INFO 12717 --- [ocp-async-35,7afd8754b67879a3,60264dc03c02e462] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:43:03.692 INFO 12717 --- [ocp-async-8,663a60d43d5c627f,707a29cecb430751] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:03.880 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,46d4d6093be42a54,c4145a923aa21eae] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=46d4d6093be42a54, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:03.880 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,46d4d6093be42a54,c4145a923aa21eae] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=46d4d6093be42a54, duration=1 ms] 2025-11-29 08:43:03.936 INFO 12717 --- [print-time-series-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Async compute queue max size = 0, current size = 0. 2025-11-29 08:43:03.987 INFO 12717 --- [ocp-schedules-30,584f9346e3190668,2deb1736fc92f2da] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog instances, size=0 2025-11-29 08:43:04.048 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,11410ec00a4f4bf8,ccd22f09aa11141e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=11410ec00a4f4bf8, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:04.053 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,11410ec00a4f4bf8,ccd22f09aa11141e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=11410ec00a4f4bf8, duration=5 ms] 2025-11-29 08:43:04.504 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:04.508 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:05.407 INFO 12717 --- [prometheus-minute-schedule-1,8f7bd4d3f53c25b4,9985b0d6e945066a] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:05.416 INFO 12717 --- [prometheus-minute-schedule-1,8f7bd4d3f53c25b4,9985b0d6e945066a] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:05.448 WARN 12717 --- [host-check-clockdiff11,d7cd6a2fce7129aa,1872b532ee7cdad4] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.48], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:05.449 ERROR 12717 --- [host-check-clockdiff11,d7cd6a2fce7129aa,1872b532ee7cdad4] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.48] 2025-11-29 08:43:05.449 WARN 12717 --- [host-check-clockdiff11,d7cd6a2fce7129aa,1872b532ee7cdad4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=12, ip=10.202.22.48, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.48 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:05.693 INFO 12717 --- [ocp-async-8,782a1c921f94f2b9,7276062b727e2283] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:05.984 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,b4492784edb7b9a8,da15dcd53a582e10] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=b4492784edb7b9a8, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:05.987 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,b4492784edb7b9a8,da15dcd53a582e10] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=b4492784edb7b9a8, duration=3 ms] 2025-11-29 08:43:06.669 INFO 12717 --- [host-check-clockdiff9,d7cd6a2fce7129aa,0a9bf0d50b2416a7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376986 -2 -2 , error= 2025-11-29 08:43:06.723 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,751deb690b5c2f53,465d59de45eda6af] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=751deb690b5c2f53, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:06.723 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,751deb690b5c2f53,465d59de45eda6af] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=751deb690b5c2f53, duration=1 ms] 2025-11-29 08:43:06.980 INFO 12717 --- [ocp-schedules-3,9ffb49a479f5aff4,6c5678370d56c6ec] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-11-29 08:43:06.992 INFO 12717 --- [ocp-schedules-3,9ffb49a479f5aff4,6c5678370d56c6ec] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:43:06.992 INFO 12717 --- [ocp-schedules-3,9ffb49a479f5aff4,6c5678370d56c6ec] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:43:07.508 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:07.509 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:07.651 INFO 12717 --- [ocp-schedules-22,4dfdc218d3eb67ee,c628a44008bcbb4c] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:43:07.654 INFO 12717 --- [ocp-schedules-22,4dfdc218d3eb67ee,c628a44008bcbb4c] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:43:07.693 INFO 12717 --- [ocp-async-1,ff715c29f73385f9,7e9adeaed6db9999] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:08.407 INFO 12717 --- [prometheus-minute-schedule-1,3dc26da9a5b16a03,9280d1a6a0ba1dcf] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:08.413 INFO 12717 --- [prometheus-minute-schedule-1,3dc26da9a5b16a03,9280d1a6a0ba1dcf] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:09.650 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,755119a7f5b7d08b,29f8d530e37d174f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=755119a7f5b7d08b, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:09.656 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,755119a7f5b7d08b,29f8d530e37d174f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=755119a7f5b7d08b, duration=7 ms] 2025-11-29 08:43:09.684 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:43:09.683937+08:00, lastProcessTime=2025-11-29T08:42:59.679630+08:00 2025-11-29 08:43:09.693 INFO 12717 --- [ocp-async-38,203d91f5f603b4ca,550767fc559a5078] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:09.928 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:43:09.683937+08:00 2025-11-29 08:43:10.015 INFO 12717 --- [ocp-schedules-19,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=3 2025-11-29 08:43:10.016 INFO 12717 --- [ocp-schedules-2,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=3 2025-11-29 08:43:10.087 INFO 12717 --- [Check obproxy2,d718317936567abc,15d8e2fccca94962] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:43:10.087 INFO 12717 --- [Check obproxy2,d718317936567abc,15d8e2fccca94962] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:10.087 INFO 12717 --- [Check obproxy1,76a5abd01aef3641,99176dcc18a5af4f] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:43:10.087 INFO 12717 --- [Collect obproxy2,8f00b7f6c5ebf18d,1baf16895eeb6044] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:43:10.087 INFO 12717 --- [Check obproxy1,76a5abd01aef3641,99176dcc18a5af4f] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:10.087 INFO 12717 --- [Collect obproxy2,8f00b7f6c5ebf18d,1baf16895eeb6044] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:10.088 INFO 12717 --- [Collect obproxy3,72fee5c24d6df018,d5f49790ff6ed366] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:43:10.088 INFO 12717 --- [Collect obproxy3,72fee5c24d6df018,d5f49790ff6ed366] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:10.091 INFO 12717 --- [Check obproxy3,2d0164b0385eacab,a639a9b0b706df93] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:43:10.091 INFO 12717 --- [Check obproxy3,2d0164b0385eacab,a639a9b0b706df93] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:10.095 INFO 12717 --- [Collect obproxy1,e3eb53eaa881dc07,8a37281c75303882] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:43:10.095 INFO 12717 --- [Collect obproxy1,e3eb53eaa881dc07,8a37281c75303882] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:10.135 INFO 12717 --- [Check obproxy2,d718317936567abc,15d8e2fccca94962] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:43:10.138 INFO 12717 --- [Check obproxy2,d718317936567abc,15d8e2fccca94962] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-11-29 08:43:10.138 INFO 12717 --- [Check obproxy2,d718317936567abc,15d8e2fccca94962] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 2, need to sync to meta: false 2025-11-29 08:43:10.138 INFO 12717 --- [Collect obproxy2,8f00b7f6c5ebf18d,1baf16895eeb6044] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:43:10.139 INFO 12717 --- [Collect obproxy3,72fee5c24d6df018,d5f49790ff6ed366] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:43:10.141 INFO 12717 --- [Collect obproxy2,8f00b7f6c5ebf18d,1baf16895eeb6044] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:43:10.143 INFO 12717 --- [Check obproxy1,76a5abd01aef3641,99176dcc18a5af4f] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:43:10.143 INFO 12717 --- [Collect obproxy3,72fee5c24d6df018,d5f49790ff6ed366] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:43:10.144 INFO 12717 --- [Collect obproxy2,8f00b7f6c5ebf18d,1baf16895eeb6044] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:43:10.146 INFO 12717 --- [Check obproxy1,76a5abd01aef3641,99176dcc18a5af4f] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-11-29 08:43:10.146 INFO 12717 --- [Check obproxy1,76a5abd01aef3641,99176dcc18a5af4f] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 1, need to sync to meta: false 2025-11-29 08:43:10.146 INFO 12717 --- [Collect obproxy3,72fee5c24d6df018,d5f49790ff6ed366] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:43:10.148 INFO 12717 --- [Check obproxy3,2d0164b0385eacab,a639a9b0b706df93] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:43:10.152 INFO 12717 --- [Check obproxy3,2d0164b0385eacab,a639a9b0b706df93] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2025-11-29 08:43:10.152 INFO 12717 --- [Check obproxy3,2d0164b0385eacab,a639a9b0b706df93] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 3 proxy_id: 3, need to sync to meta: false 2025-11-29 08:43:10.156 INFO 12717 --- [Collect obproxy1,e3eb53eaa881dc07,8a37281c75303882] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:43:10.159 INFO 12717 --- [Collect obproxy1,e3eb53eaa881dc07,8a37281c75303882] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:43:10.162 INFO 12717 --- [Collect obproxy1,e3eb53eaa881dc07,8a37281c75303882] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:43:10.199 INFO 12717 --- [ocp-schedules-11,4f068a2f69c22d4c,c513fd72029010c1] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:43:10.278 INFO 12717 --- [ocp-schedules-1,95d64e6e9f195606,267c6896e4021eac] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=2, successCount=2 2025-11-29 08:43:10.291 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,970a913cad62a856,eb72a8620afa7a58] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=970a913cad62a856, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:10.291 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,970a913cad62a856,eb72a8620afa7a58] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=970a913cad62a856, duration=1 ms] 2025-11-29 08:43:10.393 INFO 12717 --- [ocp-schedules-3,8bc80852e12c5e4f,0f595d2fb2ec0b7d] c.o.o.s.common.scheduled.PerfSchedules : Collect trans stat for alarm, time=2025-11-29T08:43:10.393+08:00, clusters=[1, 2] 2025-11-29 08:43:10.394 INFO 12717 --- [ocp-schedules-3,8bc80852e12c5e4f,0f595d2fb2ec0b7d] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster ocpserver, trans stat result is empty. 2025-11-29 08:43:10.397 INFO 12717 --- [ocp-schedules-3,8bc80852e12c5e4f,0f595d2fb2ec0b7d] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster DB, trans stat result is empty. 2025-11-29 08:43:10.509 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:10.510 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:10.527 INFO 12717 --- [ocp-schedules-14,33039a1749d759d1,275ca2366d57da58] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster zone cgroup value. 2025-11-29 08:43:10.529 INFO 12717 --- [ocp-schedules-14,33039a1749d759d1,275ca2366d57da58] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:43:10.557 INFO 12717 --- [Check cluster cgroup value.1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:10.557 INFO 12717 --- [Check cluster cgroup value.1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [enable_cgroup] 2025-11-29 08:43:10.560 INFO 12717 --- [Check cluster cgroup value.2,,] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:43:10.560 INFO 12717 --- [Check cluster cgroup value.2,,] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:43:10.560 INFO 12717 --- [Check cluster cgroup value.2,,] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:43:10.560 INFO 12717 --- [Check cluster cgroup value.2,,] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:43:10.560 INFO 12717 --- [Check cluster cgroup value.2,,] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:43:10.560 INFO 12717 --- [Check cluster cgroup value.2,,] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:43:10.560 INFO 12717 --- [Check cluster cgroup value.2,,] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:43:10.560 INFO 12717 --- [Check cluster cgroup value.2,,] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:43:10.560 INFO 12717 --- [Check cluster cgroup value.2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:10.561 INFO 12717 --- [Check cluster cgroup value.2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [enable_cgroup] 2025-11-29 08:43:10.729 INFO 12717 --- [ocp-schedules-7,b2cb7d4b9baa73fb,e012008c645547c6] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:43:10.955 INFO 12717 --- [async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 28, ruleSize = 28. 2025-11-29 08:43:11.407 INFO 12717 --- [prometheus-minute-schedule-1,2a988256a6a344e9,c59b9594101331f2] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:11.415 INFO 12717 --- [prometheus-minute-schedule-1,2a988256a6a344e9,c59b9594101331f2] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:11.694 INFO 12717 --- [ocp-async-14,4e3bbf3fafaa540c,70682f5249ef1611] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:12.602 INFO 12717 --- [ocp-schedules-29,9121770eada8fb9e,13a76c10bdc8ae21] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:43:12.607 INFO 12717 --- [alarm-detect-10,9121770eada8fb9e,63c6e0251c01c06b] c.o.o.alarm.core.detect.MetricDetector : alarmType host_agent_res_memory_over_threshold detect time range 1764376922 ~ 1764376982 matched, alarm-level: 3, evaluate result MultiExpressionEvaluateResult(metricSnapshots={host_agent_resident_memory_gb=[SeriesMetricValues(metric=Metric{name='host_agent_resident_memory_gb', labels={app=HOST, host=10.202.22.52, host_name=localhost.localdomain, ob_cluster_id=1743215594, ob_cluster_name=DB, obzone=, process=ocp_monagent, svr_ip=10.202.22.52}}, data=[ValueData(timestamp=1764376980, value=1.633)])]}, metricEvaluateResults={host_agent_resident_memory_gb > 1.5=[ExpressionEvaluateResult(matchValue=1.5, metric=host_agent_resident_memory_gb, match=true, value=1.633, values={1764376980=1.633}, valueCount=1, labels={app=HOST, host=10.202.22.52, host_name=localhost.localdomain, ob_cluster_id=1743215594, ob_cluster_name=DB, obzone=, process=ocp_monagent, svr_ip=10.202.22.52})]}, matched=true, isContainValidValue=null, message=null, level=3) 2025-11-29 08:43:12.608 INFO 12717 --- [alarm-detect-10,9121770eada8fb9e,63c6e0251c01c06b] c.o.o.a.core.detect.AlarmRuleDetector : detectByMetric done, alarmType=host_agent_res_memory_over_threshold, needSendingCount=1, detectAlarmsCount=1, matchedTargets=[alarm_template_id=3:host=10.202.22.52:process=ocp_monagent-3] 2025-11-29 08:43:12.639 WARN 12717 --- [scanner-11213814068-1,9121770eada8fb9e,1b4fb8aab8f1b593] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:43:12.674 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,7ab450963c5c4854,a12b88ce88813c7b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=7ab450963c5c4854, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:12.674 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,7ab450963c5c4854,a12b88ce88813c7b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=7ab450963c5c4854, duration=2 ms] 2025-11-29 08:43:12.898 INFO 12717 --- [metric-archive-second-24,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=1000, affectRows=1000 2025-11-29 08:43:12.899 INFO 12717 --- [metric-archive-second-17,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=1000, affectRows=1000 2025-11-29 08:43:12.903 INFO 12717 --- [metric-archive-second-24,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=993, validCount=987 2025-11-29 08:43:12.904 INFO 12717 --- [metric-archive-second-17,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=995, validCount=995 2025-11-29 08:43:12.947 INFO 12717 --- [metric-archive-second-10,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=1000, affectRows=1000 2025-11-29 08:43:12.952 INFO 12717 --- [metric-archive-second-10,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=997, validCount=997 2025-11-29 08:43:12.967 INFO 12717 --- [metric-archive-second-22,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=1000, affectRows=1000 2025-11-29 08:43:12.973 INFO 12717 --- [metric-archive-second-22,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=985, validCount=984 2025-11-29 08:43:12.989 INFO 12717 --- [metric-archive-second-15,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=1000, affectRows=1000 2025-11-29 08:43:12.995 INFO 12717 --- [metric-archive-second-15,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=998, validCount=995 2025-11-29 08:43:13.057 INFO 12717 --- [metric-archive-second-18,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=1000, affectRows=1000 2025-11-29 08:43:13.062 INFO 12717 --- [metric-archive-second-18,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=993, validCount=991 2025-11-29 08:43:13.097 INFO 12717 --- [ocp-schedules-17,741c101c86f47ab9,83daabf75415358b] c.o.o.s.c.scheduled.TenantSchedules : Check all tenant service name conflict 2025-11-29 08:43:13.098 INFO 12717 --- [ocp-schedules-17,741c101c86f47ab9,83daabf75415358b] .t.CheckAllTenantServiceNameConflictTask : Check all tenant service name conflict. 2025-11-29 08:43:13.116 INFO 12717 --- [metric-archive-second-26,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=621, affectRows=621 2025-11-29 08:43:13.120 INFO 12717 --- [metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=747, affectRows=747 2025-11-29 08:43:13.120 INFO 12717 --- [metric-archive-second-26,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=610, validCount=607 2025-11-29 08:43:13.120 INFO 12717 --- [metric-archive-second-2,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=1000, affectRows=1000 2025-11-29 08:43:13.124 INFO 12717 --- [metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=742, validCount=737 2025-11-29 08:43:13.125 INFO 12717 --- [metric-archive-second-2,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=995, validCount=993 2025-11-29 08:43:13.129 INFO 12717 --- [ocp-async-23,e69a5da6e6199067,10baee4db7601c1d] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:43:13.149 INFO 12717 --- [metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=718, affectRows=718 2025-11-29 08:43:13.152 INFO 12717 --- [metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=707, validCount=704 2025-11-29 08:43:13.393 INFO 12717 --- [metric-archive-second-11,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376920, nodeCount=1000, affectRows=1000 2025-11-29 08:43:13.398 INFO 12717 --- [metric-archive-second-11,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376920, nodeCount=999, validCount=997 2025-11-29 08:43:13.510 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:13.511 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:13.620 INFO 12717 --- [host-check-clockdiff8,d7cd6a2fce7129aa,ab9d606f5bab54a9] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764376993 -5 -5 , error= 2025-11-29 08:43:13.693 INFO 12717 --- [ocp-async-22,3cde7826e2ced6d1,d15285558e47f869] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:14.240 INFO 12717 --- [metric-archive-async-18,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1764376920, nodeCount=354, affectRows=354 2025-11-29 08:43:14.407 INFO 12717 --- [prometheus-minute-schedule-1,faa778ba9d4bdcd7,dd41e9a90fa58a89] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:14.417 INFO 12717 --- [prometheus-minute-schedule-1,faa778ba9d4bdcd7,dd41e9a90fa58a89] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:14.530 INFO 12717 --- [metric-archive-async-27,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1764376920, nodeCount=1000, affectRows=1000 2025-11-29 08:43:15.040 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,e250a926a4e2df89,69bbaf07a461f868] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=e250a926a4e2df89, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:15.040 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,e250a926a4e2df89,69bbaf07a461f868] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=e250a926a4e2df89, duration=1 ms] 2025-11-29 08:43:15.287 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,6c1a654856b4def4,9b2a3dc11254fbbe] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=6c1a654856b4def4, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:15.291 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,6c1a654856b4def4,9b2a3dc11254fbbe] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=6c1a654856b4def4, duration=5 ms] 2025-11-29 08:43:15.693 INFO 12717 --- [ocp-async-22,a9f75e73ff895345,d5759f6fc67e31df] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:16.377 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,9761df2ea33ed370,97d59e710b1ad406] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=9761df2ea33ed370, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:16.379 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,9761df2ea33ed370,97d59e710b1ad406] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=9761df2ea33ed370, duration=2 ms] 2025-11-29 08:43:16.511 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:16.514 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:17.407 INFO 12717 --- [prometheus-minute-schedule-1,b0a39074ccdbb454,6a0b0c4d5164c2b0] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:17.414 INFO 12717 --- [prometheus-minute-schedule-1,b0a39074ccdbb454,6a0b0c4d5164c2b0] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:17.694 INFO 12717 --- [ocp-async-17,82081de0f02e579d,f508dd2922f806f4] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:17.799 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,b4b986aa7489c9cb,673622480112fdb4] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=b4b986aa7489c9cb, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:17.799 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,b4b986aa7489c9cb,673622480112fdb4] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=b4b986aa7489c9cb, duration=1 ms] 2025-11-29 08:43:17.815 INFO 12717 --- [ocp-schedules-29,9c671da2517bb162,26878a6f058c2747] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:43:17.818 INFO 12717 --- [ocp-schedules-29,9c671da2517bb162,26878a6f058c2747] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:43:18.353 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.m.helper.ExporterRequestHelper : Refresh exporter helper 2025-11-29 08:43:18.374 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:43:18.380 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:43:18.602 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:43:18.604 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:43:18.816 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:43:18.817 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:43:18.917 INFO 12717 --- [ocp-schedules-18,95810146d183bf67,2d60ad2c11ad9ff2] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-11-29 08:43:19.042 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:43:19.044 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:43:19.246 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:43:19.248 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:43:19.362 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:43:19.364 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:43:19.388 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:43:19.389 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:43:19.408 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:43:19.409 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:43:19.432 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:43:19.434 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:43:19.454 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:43:19.455 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:43:19.474 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:43:19.476 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:43:19.494 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:43:19.495 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:43:19.512 INFO 12717 --- [ocp-schedules-27,98d26ba305f417d8,2e18b1f91740a6cb] c.o.o.m.helper.ExporterRequestHelper : Refresh exporter helper finished, credentialSize=12, cachedSize=13 2025-11-29 08:43:19.514 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:19.517 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:19.688 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:43:19.687670+08:00, lastProcessTime=2025-11-29T08:43:09.683937+08:00 2025-11-29 08:43:19.694 INFO 12717 --- [ocp-async-32,5f9ec7a406a924fb,503aa994a3515101] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:19.908 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : aggregate done, channelId=1, alarmsCount=2, resultsCount=2 2025-11-29 08:43:20.012 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:43:19.687670+08:00 2025-11-29 08:43:20.208 INFO 12717 --- [ocp-schedules-13,470d810ca317250b,ad4a47cbdde91069] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:43:20.308 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-77,5505947ef77ef7ad,5c0db5633e574df4] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=5505947ef77ef7ad, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:20.313 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-77,5505947ef77ef7ad,5c0db5633e574df4] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=5505947ef77ef7ad, duration=6 ms] 2025-11-29 08:43:20.407 INFO 12717 --- [prometheus-minute-schedule-1,6c993a8330a88b79,5d3118c6d9c804d1] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:20.414 INFO 12717 --- [prometheus-minute-schedule-1,6c993a8330a88b79,5d3118c6d9c804d1] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:21.467 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-100,8cb2b596d7c78b6f,d94805d7fda2723b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=8cb2b596d7c78b6f, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:21.468 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-100,8cb2b596d7c78b6f,d94805d7fda2723b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=8cb2b596d7c78b6f, duration=1 ms] 2025-11-29 08:43:21.694 INFO 12717 --- [ocp-async-1,979aa911fc7e536f,eeb3bcd3e7aef21e] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:22.499 INFO 12717 --- [ocp-schedules-20,5043758a298237c0,a2a8aeeba16251d4] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":40355,"minuteWriteQueue":0,"minuteReadCacheSize":17215,"secondWriteCacheSize":10086,"seriesIdKeySize":17387,"secondWriteQueue":0,"minuteWriteCacheSize":17216} 2025-11-29 08:43:22.499 INFO 12717 --- [ocp-schedules-20,5043758a298237c0,a2a8aeeba16251d4] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{ob_sysstat_40012=49, ob_server_resource_memory_assigned_bytes=23, ob_sysstat_40011=49, ob_sysstat_40010=49, ob_unit_num=23, ob_database_disk_required_size=74, go_memstats_mspan_sys_bytes=25, ob_sysstat_30213=42, go_memstats_stack_sys_bytes=25, ob_sysstat_30212=42, ob_sysstat_30217=42, ob_sysstat_30216=42, ob_tenant_request_queue_total_size=72, jvm_gc_memory_allocated_bytes_total=2, ob_system_event_total_waits=59, ob_tenant_disk_v4_log_disk_size=40, ob_sysstat_40019=49, ob_sysstat_40018=49, processor_process_metrics_total=30, jvm_gc_max_data_size_bytes=2, go_memstats_frees_total=25, host_available=29, ob_sysstat_30220=42, ob_sysstat_30222=42, ob_sysstat_30221=42, mysql_output_write_sql_failed_count=29, ob_database_disk_data_size=73, ob_tenant_memstore_freeze_cnt=40, ob_tenant_unit_min_memory=96, ob_sysstat_60087=49, ob_memstore_total_bytes=72, jvm_threads_states_threads=8, process_resident_memory_bytes=25, ob_cache_size_bytes=67, ob_tenant_unit_max_memory=96, input_collect_metrics_total=34, ob_sysstat_50039=50, ob_sysstat_60083=50, ob_sysstat_50038=49, node_disk_io_time_weighted_seconds_total=39, process_open_fds=25, ob_sysstat_50037=49, process_memory_rss_bytes=41, ob_server_resource_cpu_assigned_percent=23, ob_sysstat_50030=50, sql_audit_input_collect_lost_total=36, go_memstats_heap_inuse_bytes=25, ob_sysstat_40118=49, ob_sysstat_40117=49, node_memory_memfree_bytes=32, ob_sysstat_40116=49, ob_sysstat_40005=49, process_exists=50, ob_sysstat_40004=49, ob_sysstat_40003=49, ob_sysstat_40002=49, go_gc_duration_seconds=30, ob_server_resource_disk_bytes=23, ob_sysstat_40001=49, ob_sysstat_40000=49, mysql_output_write_sql_time=33, ob_sysstat_30200=42, odp_current_session=36, ob_sysstat_30201=42, ob_sysstat_30204=42, ob_sysstat_30205=42, ob_sysstat_30208=42, process_thread_count=41, ob_sysstat_30209=42, ob_sysstat_50040=50, ob_sysstat_40009=49, ob_disk_free_bytes=23, ob_sysstat_40008=49, node_filesystem_avail_bytes=67, ob_sysstat_40007=49, ob_sysstat_40006=49, node_time_seconds=31, ob_unit_config_max_i 2025-11-29 08:43:22.517 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:22.518 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:22.754 INFO 12717 --- [ocp-schedules-7,cd4f941306ee42e6,5327c7f56d6b6e57] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:43:22.761 INFO 12717 --- [alarm-detect-19,cd4f941306ee42e6,1bce65cebb33d084] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008327, data is empty. 2025-11-29 08:43:22.765 INFO 12717 --- [alarm-detect-8,cd4f941306ee42e6,c6a866c5c0fdba16] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008327, data is empty. 2025-11-29 08:43:22.766 INFO 12717 --- [alarm-detect-19,cd4f941306ee42e6,1bce65cebb33d084] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008327, data is empty. 2025-11-29 08:43:22.766 INFO 12717 --- [alarm-detect-18,cd4f941306ee42e6,148848f43d7c2585] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008327, data is empty. 2025-11-29 08:43:22.772 INFO 12717 --- [alarm-detect-8,cd4f941306ee42e6,c6a866c5c0fdba16] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008327, data is empty. 2025-11-29 08:43:22.773 INFO 12717 --- [alarm-detect-18,cd4f941306ee42e6,148848f43d7c2585] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008327, data is empty. 2025-11-29 08:43:22.773 WARN 12717 --- [scanner-11213814278-1,cd4f941306ee42e6,4c2b2c523dac1dd2] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:43:22.818 INFO 12717 --- [alarm-detect-9,cd4f941306ee42e6,745d71dccc6f9649] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008327, data is empty. 2025-11-29 08:43:22.822 INFO 12717 --- [alarm-detect-9,cd4f941306ee42e6,745d71dccc6f9649] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008327, data is empty. 2025-11-29 08:43:23.129 INFO 12717 --- [ocp-async-8,1e8bbc6666feb879,a1958a2181f85115] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:43:23.337 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-98,10922f1b1f2d42ed,428cd1dcd72eeab9] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=10922f1b1f2d42ed, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:23.337 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-98,10922f1b1f2d42ed,428cd1dcd72eeab9] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=10922f1b1f2d42ed, duration=1 ms] 2025-11-29 08:43:23.407 INFO 12717 --- [prometheus-minute-schedule-1,380e2458f57d3a28,0d23506868474682] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:23.414 INFO 12717 --- [prometheus-minute-schedule-1,380e2458f57d3a28,0d23506868474682] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:23.694 INFO 12717 --- [ocp-async-35,2414208d460c644e,8801516a8ef8c5c7] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:25.154 INFO 12717 --- [ocp-schedules-20,3f009c47aaae8137,a6eacc33541fba30] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info 2025-11-29 08:43:25.168 INFO 12717 --- [ocp-schedules-20,3f009c47aaae8137,a6eacc33541fba30] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Invalidate clusters [] cache info success. 2025-11-29 08:43:25.168 INFO 12717 --- [ocp-schedules-20,3f009c47aaae8137,a6eacc33541fba30] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info success, cached size: 2 2025-11-29 08:43:25.302 INFO 12717 --- [ocp-schedules-4,351488c6ddcbee74,11ed1e2543d6f857] c.o.o.s.common.scheduled.PerfSchedules : Oas sql curing, clusters=[] 2025-11-29 08:43:25.306 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,c4e609b19c1453a2,da393a776f3c6205] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=c4e609b19c1453a2, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:25.306 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,c4e609b19c1453a2,da393a776f3c6205] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=c4e609b19c1453a2, duration=1 ms] 2025-11-29 08:43:25.424 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,522a4bb6815a814d,4aad904509d8b6d2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=522a4bb6815a814d, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:25.429 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,522a4bb6815a814d,4aad904509d8b6d2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=522a4bb6815a814d, duration=6 ms] 2025-11-29 08:43:25.520 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:25.522 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:25.694 INFO 12717 --- [ocp-async-14,2053c085913146c9,b27bfe0ce0ec6357] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:26.407 INFO 12717 --- [prometheus-minute-schedule-1,26544ea39580fdc6,0e8d1afea297df37] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:26.415 INFO 12717 --- [prometheus-minute-schedule-1,26544ea39580fdc6,0e8d1afea297df37] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:26.747 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,2bfb5e6b3813c066,ffdb5a2025dd5d8a] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=2bfb5e6b3813c066, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:26.748 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,2bfb5e6b3813c066,ffdb5a2025dd5d8a] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=2bfb5e6b3813c066, duration=2 ms] 2025-11-29 08:43:26.997 INFO 12717 --- [ocp-schedules-15,b0fafba7a20c9af3,15997ea7cf90a820] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-11-29 08:43:27.011 INFO 12717 --- [ocp-schedules-15,b0fafba7a20c9af3,15997ea7cf90a820] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:43:27.011 INFO 12717 --- [ocp-schedules-15,b0fafba7a20c9af3,15997ea7cf90a820] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:43:27.694 INFO 12717 --- [ocp-async-6,e733074921a9c18d,fb20c205f6055e96] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:28.018 INFO 12717 --- [ocp-schedules-20,9cae2a4dc801cd20,0ebb68209c24d00e] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:43:28.022 INFO 12717 --- [ocp-schedules-20,9cae2a4dc801cd20,0ebb68209c24d00e] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:43:28.064 INFO 12717 --- [ocp-schedules-12,c7b1e13849e043a1,d7a296a9fce5677d] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:43:28.064 INFO 12717 --- [ocp-schedules-12,e1a133129b9354e2,b5df8aafb4ff921e] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-11-29 08:43:28.066 INFO 12717 --- [ocp-schedules-12,e1a133129b9354e2,b5df8aafb4ff921e] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:43:28.119 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:28.120 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:43:28.126 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:28.126 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:43:28.158 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:28.159 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:28.159 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:43:28.190 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:28.191 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:28.191 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:43:28.192 WARN 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=OCP 2025-11-29 08:43:28.192 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:28.195 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:43:28.195 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:43:28.205 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.205 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:28.346 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:28.347 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:28.348 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:43:28.391 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.401 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:28.401 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:28.402 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:43:28.403 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster ocpserver 2025-11-29 08:43:28.431 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2025-11-29 08:43:28.460 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:43:28.466 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:43:28.508 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:28.509 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:28.509 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:43:28.513 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:28.516 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:43:28.520 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:43:28.523 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:28.523 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:28.536 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.536 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:28.648 INFO 12717 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:43:28.751 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.751 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:12+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:28.766 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=8073, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:43:28.776 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:28.777 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:28.777 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:43:28.779 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.779 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:28.787 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=ocpserver, obClusterId=1743215593 2025-11-29 08:43:28.787 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=ocpserver, obClusterId=1743215593, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"ocpserver","ObClusterId":1743215593,"ObRegion":"ocpserver","ObRegionId":1743215593,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.58:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376673675000}, createTime=2025-03-29T10:43:06+08:00, creator=admin, obVersion=4.2.1.8, packageOperatingSystem=null, loadType=null, tenantCount=3) 2025-11-29 08:43:28.795 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.795 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:43:28.809 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.809 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:28.824 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.824 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:28.840 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.840 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:28.855 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.855 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:30+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:28.870 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.870 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:28.885 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.885 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:43:28.897 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.897 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:28.912 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:28.913 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:42+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:29.119 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:29.119 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:29.137 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:29.151 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:29.152 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:29.153 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:43:29.313 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster DB 2025-11-29 08:43:29.344 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 2025-11-29 08:43:29.349 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:43:29.355 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:43:29.394 INFO 12717 --- [Sync all clusters2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:43:29.407 INFO 12717 --- [prometheus-minute-schedule-1,eb119f3d08af2f6c,23729029c9311fcb] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:29.420 INFO 12717 --- [prometheus-minute-schedule-1,eb119f3d08af2f6c,23729029c9311fcb] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:29.553 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 got ObProcessInfoResult(pid=2447, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:43:29.566 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:29.567 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:29.568 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:43:29.583 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=DB, obClusterId=1743215594 2025-11-29 08:43:29.583 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=DB, obClusterId=1743215594, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=CREATED, syncStatus=VALID, rootserverJson={"ObCluster":"DB","ObClusterId":1743215594,"ObRegion":"DB","ObRegionId":1743215594,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.52:2882","role":"LEADER","sql_port":2881},{"address":"10.202.22.49:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.202.22.61:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376629943000}, createTime=2025-03-29T11:15:04+08:00, creator=admin, obVersion=4.3.5.1, packageOperatingSystem=el8, loadType=HTAP, tenantCount=7) 2025-11-29 08:43:29.682 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:43:29.682058+08:00, lastProcessTime=2025-11-29T08:43:19.687670+08:00 2025-11-29 08:43:29.695 INFO 12717 --- [ocp-async-33,2b2c6878e63b42da,f489287bdb60fa8c] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:29.910 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:43:29.682058+08:00 2025-11-29 08:43:30.213 INFO 12717 --- [ocp-schedules-30,df16d06dd49defba,620049ec3abdba14] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:43:30.261 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-89,3f530896426c3c7c,7f8a8d5a816d982e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=3f530896426c3c7c, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:30.261 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-89,3f530896426c3c7c,7f8a8d5a816d982e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=3f530896426c3c7c, duration=1 ms] 2025-11-29 08:43:30.271 INFO 12717 --- [ocp-schedules-23,a2dfab30cf4fe82a,300dafbba2db4591] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-11-29 08:43:30.271 INFO 12717 --- [ocp-schedules-27,f01a2e37dd41934c,ad94f46d40a1135d] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-11-29 08:43:30.271 INFO 12717 --- [ocp-schedules-11,ab98e978e0619e5e,31053f3f80d0febc] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-11-29 08:43:30.274 INFO 12717 --- [ocp-schedules-27,f01a2e37dd41934c,ad94f46d40a1135d] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:43:30.274 INFO 12717 --- [ocp-schedules-11,ab98e978e0619e5e,31053f3f80d0febc] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:43:30.288 INFO 12717 --- [ocp-schedules-24,1651b361a5d294c5,ea358fb78eda982a] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-11-29 08:43:30.302 INFO 12717 --- [ocp-schedules-13,883683064d2ddd8d,a22077d946d65ee9] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:43:30.303 INFO 12717 --- [host-check-agent-status1,883683064d2ddd8d,f158f8bf6915ff44] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:43:30.303 INFO 12717 --- [host-check-agent-status2,883683064d2ddd8d,e8c4f3fb1b08a031] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:43:30.303 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:30.303 INFO 12717 --- [host-check-agent-status3,883683064d2ddd8d,621b8fbcb657de98] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:43:30.303 INFO 12717 --- [host-check-agent-status4,883683064d2ddd8d,671354524a9e1161] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:43:30.303 INFO 12717 --- [host-check-agent-status5,883683064d2ddd8d,00e753776d154511] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:43:30.303 INFO 12717 --- [ocp-schedules-15,63a56589a6ba43b2,6297cdd3e849bae2] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:43:30.303 INFO 12717 --- [host-check-agent-status6,883683064d2ddd8d,abb2ade9eafeeebc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:43:30.303 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:43:30.304 INFO 12717 --- [host-check-agent-status7,883683064d2ddd8d,690ce7413aff633c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:43:30.304 INFO 12717 --- [host-check-agent-status8,883683064d2ddd8d,ac8c22ead25008c5] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:43:30.304 INFO 12717 --- [host-check-agent-status9,883683064d2ddd8d,dcb0a44e7b943ee0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:43:30.304 INFO 12717 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.8 2025-11-29 08:43:30.304 INFO 12717 --- [host-check-agent-status10,883683064d2ddd8d,e4e5d42c7d22d9c3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:43:30.304 INFO 12717 --- [host-check-clockdiff1,63a56589a6ba43b2,190c0d0e52575c1c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.58], timeoutMillis=20000 2025-11-29 08:43:30.305 INFO 12717 --- [host-check-agent-status11,883683064d2ddd8d,44bf519af34e6db7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:43:30.305 INFO 12717 --- [host-check-agent-status12,883683064d2ddd8d,43a2fc283c09bd27] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:43:30.305 INFO 12717 --- [host-check-clockdiff2,63a56589a6ba43b2,e693527531dc81f5] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.50], timeoutMillis=20000 2025-11-29 08:43:30.305 INFO 12717 --- [host-check-clockdiff3,63a56589a6ba43b2,d0ab82b80f473517] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.51], timeoutMillis=20000 2025-11-29 08:43:30.305 INFO 12717 --- [host-check-clockdiff4,63a56589a6ba43b2,e089badbeecfbf09] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.52], timeoutMillis=20000 2025-11-29 08:43:30.305 INFO 12717 --- [host-check-clockdiff5,63a56589a6ba43b2,9c3090e743fc6efa] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.53], timeoutMillis=20000 2025-11-29 08:43:30.306 INFO 12717 --- [host-check-clockdiff6,63a56589a6ba43b2,479d553a63557e0c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.54], timeoutMillis=20000 2025-11-29 08:43:30.306 INFO 12717 --- [host-check-clockdiff7,63a56589a6ba43b2,5bbd9b0684f2485b] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.59], timeoutMillis=20000 2025-11-29 08:43:30.306 INFO 12717 --- [host-check-clockdiff8,63a56589a6ba43b2,89eb14c05322b744] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.60], timeoutMillis=20000 2025-11-29 08:43:30.306 INFO 12717 --- [host-check-clockdiff9,63a56589a6ba43b2,80ecb236d418dacf] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.61], timeoutMillis=20000 2025-11-29 08:43:30.306 INFO 12717 --- [host-check-clockdiff10,63a56589a6ba43b2,d2721280d9c28591] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.47], timeoutMillis=20000 2025-11-29 08:43:30.307 INFO 12717 --- [host-check-clockdiff11,63a56589a6ba43b2,d5ac131940f4f9d5] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.48], timeoutMillis=20000 2025-11-29 08:43:30.307 INFO 12717 --- [host-check-clockdiff12,63a56589a6ba43b2,3aee8aad60223e89] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.49], timeoutMillis=20000 2025-11-29 08:43:30.308 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:30.309 INFO 12717 --- [host-check-agent-status1,883683064d2ddd8d,f158f8bf6915ff44] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:43:30.309 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:43:30.313 INFO 12717 --- [host-check-agent-status5,883683064d2ddd8d,00e753776d154511] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:43:30.313 INFO 12717 --- [host-check-agent-status7,883683064d2ddd8d,690ce7413aff633c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:43:30.314 INFO 12717 --- [ocp-schedules-26,0c05aa171b6fe023,6ea9c9b0130244c1] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:43:30.314 INFO 12717 --- [host-check-agent-status2,883683064d2ddd8d,e8c4f3fb1b08a031] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:43:30.314 INFO 12717 --- [host-check-agent-status1,883683064d2ddd8d,f158f8bf6915ff44] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:43:30.315 INFO 12717 --- [host-check-agent-status9,883683064d2ddd8d,dcb0a44e7b943ee0] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:43:30.315 INFO 12717 --- [host-check-agent-status3,883683064d2ddd8d,621b8fbcb657de98] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:43:30.315 INFO 12717 --- [host-check-agent-status7,883683064d2ddd8d,690ce7413aff633c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:43:30.315 INFO 12717 --- [host-check-agent-status11,883683064d2ddd8d,44bf519af34e6db7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:43:30.315 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:43:30.316 INFO 12717 --- [host-check-agent-status4,883683064d2ddd8d,671354524a9e1161] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:43:30.316 INFO 12717 --- [host-check-agent-status6,883683064d2ddd8d,abb2ade9eafeeebc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:43:30.316 INFO 12717 --- [host-check-agent-status12,883683064d2ddd8d,43a2fc283c09bd27] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:43:30.316 INFO 12717 --- [host-check-agent-status10,883683064d2ddd8d,e4e5d42c7d22d9c3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:43:30.316 INFO 12717 --- [host-check-agent-status9,883683064d2ddd8d,dcb0a44e7b943ee0] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:43:30.316 INFO 12717 --- [host-check-agent-status5,883683064d2ddd8d,00e753776d154511] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:43:30.317 INFO 12717 --- [host-check-agent-status11,883683064d2ddd8d,44bf519af34e6db7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:43:30.318 INFO 12717 --- [host-check-agent-status2,883683064d2ddd8d,e8c4f3fb1b08a031] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:43:30.318 INFO 12717 --- [host-check-agent-status3,883683064d2ddd8d,621b8fbcb657de98] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:43:30.318 INFO 12717 --- [host-check-agent-status6,883683064d2ddd8d,abb2ade9eafeeebc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:43:30.318 INFO 12717 --- [host-check-agent-status8,883683064d2ddd8d,ac8c22ead25008c5] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:43:30.318 INFO 12717 --- [host-check-agent-status12,883683064d2ddd8d,43a2fc283c09bd27] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:43:30.318 INFO 12717 --- [host-check-agent-status10,883683064d2ddd8d,e4e5d42c7d22d9c3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:43:30.321 INFO 12717 --- [host-check-agent-status4,883683064d2ddd8d,671354524a9e1161] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:43:30.321 INFO 12717 --- [host-check-agent-status8,883683064d2ddd8d,ac8c22ead25008c5] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:43:30.326 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:43:30.328 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:43:30.340 INFO 12717 --- [host-check-clockdiff1,63a56589a6ba43b2,190c0d0e52575c1c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764377010 0 0 , error= 2025-11-29 08:43:30.340 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:43:30.341 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:43:30.341 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:43:30.341 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:43:30.341 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:43:30.342 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:43:30.342 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:43:30.342 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:43:30.342 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:43:30.343 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:43:30.343 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:43:30.349 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:43:30.352 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:43:30.353 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:43:30.354 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:43:30.354 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:43:30.354 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:43:30.354 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:43:30.354 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:43:30.354 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:43:30.354 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:43:30.355 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:43:30.355 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:43:30.355 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:43:30.355 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:43:30.356 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:43:30.356 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:43:30.356 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:43:30.356 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:43:30.356 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:43:30.357 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:43:30.357 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:43:30.358 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:43:30.364 INFO 12717 --- [host-check-agent-status9,883683064d2ddd8d,dcb0a44e7b943ee0] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.365 INFO 12717 --- [host-check-agent-status12,883683064d2ddd8d,43a2fc283c09bd27] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.366 INFO 12717 --- [host-check-agent-status3,883683064d2ddd8d,621b8fbcb657de98] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.366 INFO 12717 --- [host-check-agent-status6,883683064d2ddd8d,abb2ade9eafeeebc] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.366 INFO 12717 --- [host-check-agent-status1,883683064d2ddd8d,f158f8bf6915ff44] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.366 INFO 12717 --- [host-check-agent-status2,883683064d2ddd8d,e8c4f3fb1b08a031] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.367 INFO 12717 --- [host-check-agent-status7,883683064d2ddd8d,690ce7413aff633c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.368 INFO 12717 --- [host-check-agent-status10,883683064d2ddd8d,e4e5d42c7d22d9c3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.369 INFO 12717 --- [host-check-agent-status1,883683064d2ddd8d,f158f8bf6915ff44] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.58 2025-11-29 08:43:30.369 INFO 12717 --- [host-check-agent-status2,883683064d2ddd8d,e8c4f3fb1b08a031] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.50 2025-11-29 08:43:30.369 INFO 12717 --- [host-check-agent-status2,883683064d2ddd8d,e8c4f3fb1b08a031] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.50 2025-11-29 08:43:30.369 INFO 12717 --- [host-check-agent-status1,883683064d2ddd8d,f158f8bf6915ff44] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.58 2025-11-29 08:43:30.369 INFO 12717 --- [host-check-agent-status2,883683064d2ddd8d,e8c4f3fb1b08a031] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.50 2025-11-29 08:43:30.369 INFO 12717 --- [host-check-agent-status1,883683064d2ddd8d,f158f8bf6915ff44] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.58 2025-11-29 08:43:30.369 INFO 12717 --- [host-check-agent-status5,883683064d2ddd8d,00e753776d154511] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.370 INFO 12717 --- [host-check-agent-status9,883683064d2ddd8d,dcb0a44e7b943ee0] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.61 2025-11-29 08:43:30.370 INFO 12717 --- [host-check-agent-status9,883683064d2ddd8d,dcb0a44e7b943ee0] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.61 2025-11-29 08:43:30.370 INFO 12717 --- [host-check-agent-status9,883683064d2ddd8d,dcb0a44e7b943ee0] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.61 2025-11-29 08:43:30.370 INFO 12717 --- [host-check-agent-status6,883683064d2ddd8d,abb2ade9eafeeebc] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.54 2025-11-29 08:43:30.370 INFO 12717 --- [host-check-agent-status6,883683064d2ddd8d,abb2ade9eafeeebc] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.54 2025-11-29 08:43:30.370 INFO 12717 --- [host-check-agent-status6,883683064d2ddd8d,abb2ade9eafeeebc] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.54 2025-11-29 08:43:30.370 INFO 12717 --- [host-check-agent-status12,883683064d2ddd8d,43a2fc283c09bd27] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.49 2025-11-29 08:43:30.370 INFO 12717 --- [host-check-agent-status12,883683064d2ddd8d,43a2fc283c09bd27] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.49 2025-11-29 08:43:30.370 INFO 12717 --- [host-check-agent-status12,883683064d2ddd8d,43a2fc283c09bd27] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.49 2025-11-29 08:43:30.371 INFO 12717 --- [host-check-agent-status8,883683064d2ddd8d,ac8c22ead25008c5] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.372 INFO 12717 --- [host-check-agent-status3,883683064d2ddd8d,621b8fbcb657de98] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.51 2025-11-29 08:43:30.372 INFO 12717 --- [host-check-agent-status3,883683064d2ddd8d,621b8fbcb657de98] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.51 2025-11-29 08:43:30.372 INFO 12717 --- [host-check-agent-status3,883683064d2ddd8d,621b8fbcb657de98] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.51 2025-11-29 08:43:30.372 INFO 12717 --- [host-check-agent-status4,883683064d2ddd8d,671354524a9e1161] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.373 INFO 12717 --- [host-check-agent-status11,883683064d2ddd8d,44bf519af34e6db7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/agent/status, params:null 2025-11-29 08:43:30.374 INFO 12717 --- [host-check-agent-status10,883683064d2ddd8d,e4e5d42c7d22d9c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.47 2025-11-29 08:43:30.374 INFO 12717 --- [host-check-agent-status10,883683064d2ddd8d,e4e5d42c7d22d9c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.47 2025-11-29 08:43:30.374 INFO 12717 --- [host-check-agent-status10,883683064d2ddd8d,e4e5d42c7d22d9c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.47 2025-11-29 08:43:30.375 INFO 12717 --- [host-check-agent-status8,883683064d2ddd8d,ac8c22ead25008c5] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.60 2025-11-29 08:43:30.375 INFO 12717 --- [host-check-agent-status8,883683064d2ddd8d,ac8c22ead25008c5] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.60 2025-11-29 08:43:30.375 INFO 12717 --- [host-check-agent-status8,883683064d2ddd8d,ac8c22ead25008c5] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.60 2025-11-29 08:43:30.376 INFO 12717 --- [host-check-agent-status4,883683064d2ddd8d,671354524a9e1161] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.52 2025-11-29 08:43:30.377 INFO 12717 --- [host-check-agent-status4,883683064d2ddd8d,671354524a9e1161] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.52 2025-11-29 08:43:30.377 INFO 12717 --- [host-check-agent-status4,883683064d2ddd8d,671354524a9e1161] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.52 2025-11-29 08:43:30.377 INFO 12717 --- [host-check-agent-status5,883683064d2ddd8d,00e753776d154511] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.53 2025-11-29 08:43:30.377 INFO 12717 --- [host-check-agent-status5,883683064d2ddd8d,00e753776d154511] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.53 2025-11-29 08:43:30.377 INFO 12717 --- [host-check-agent-status5,883683064d2ddd8d,00e753776d154511] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.53 2025-11-29 08:43:30.379 INFO 12717 --- [host-check-agent-status7,883683064d2ddd8d,690ce7413aff633c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.59 2025-11-29 08:43:30.380 INFO 12717 --- [host-check-agent-status7,883683064d2ddd8d,690ce7413aff633c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.59 2025-11-29 08:43:30.380 INFO 12717 --- [host-check-agent-status7,883683064d2ddd8d,690ce7413aff633c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.59 2025-11-29 08:43:30.380 INFO 12717 --- [Check status for all clusters2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.3.5.1 2025-11-29 08:43:30.382 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/time, params:null 2025-11-29 08:43:30.383 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.202.22.58, timestamp=2025-11-29T08:43:30.383538801+08:00 2025-11-29 08:43:30.384 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/time, params:null 2025-11-29 08:43:30.384 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-11-29T08:43:30.384+08:00 2025-11-29 08:43:30.385 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.202.22.50, timestamp=2025-11-29T08:43:30.385069331+08:00 2025-11-29 08:43:30.385 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-11-29T08:43:30.385+08:00 2025-11-29 08:43:30.385 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/time, params:null 2025-11-29 08:43:30.386 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/time, params:null 2025-11-29 08:43:30.386 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/time, params:null 2025-11-29 08:43:30.387 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/time, params:null 2025-11-29 08:43:30.387 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.202.22.52, timestamp=2025-11-29T08:43:30.385177894+08:00 2025-11-29 08:43:30.387 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.202.22.54, timestamp=2025-11-29T08:43:30.385318947+08:00 2025-11-29 08:43:30.388 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-11-29T08:43:30.387+08:00 2025-11-29 08:43:30.388 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/time, params:null 2025-11-29 08:43:30.388 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2025-11-29T08:43:30.387+08:00 2025-11-29 08:43:30.388 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-11-29T08:43:30.384+08:00 and status AVAILABLE 2025-11-29 08:43:30.388 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/time, params:null 2025-11-29 08:43:30.388 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2025-11-29T08:43:30.385+08:00 and status AVAILABLE 2025-11-29 08:43:30.388 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=13, ip=10.202.22.49, timestamp=2025-11-29T08:43:30.386639176+08:00 2025-11-29 08:43:30.389 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=13, lastAvailableTime=2025-11-29T08:43:30.388+08:00 2025-11-29 08:43:30.389 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=8, ip=10.202.22.59, timestamp=2025-11-29T08:43:30.388128698+08:00 2025-11-29 08:43:30.389 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/time, params:null 2025-11-29 08:43:30.390 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-11-29T08:43:30.387+08:00 and status AVAILABLE 2025-11-29 08:43:30.390 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=8, lastAvailableTime=2025-11-29T08:43:30.389+08:00 2025-11-29 08:43:30.391 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/time, params:null 2025-11-29 08:43:30.391 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=12, ip=10.202.22.48, timestamp=2025-11-29T08:43:30.389137381+08:00 2025-11-29 08:43:30.391 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-11-29T08:43:30.387+08:00 and status AVAILABLE 2025-11-29 08:43:30.392 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=12, lastAvailableTime=2025-11-29T08:43:30.391+08:00 2025-11-29 08:43:30.392 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.s.compute.HostAgentDaoManager : Updated agent 12 last available time 2025-11-29T08:43:30.388+08:00 and status AVAILABLE 2025-11-29 08:43:30.393 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.202.22.53, timestamp=2025-11-29T08:43:30.391500905+08:00 2025-11-29 08:43:30.393 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.s.compute.HostAgentDaoManager : Updated agent 8 last available time 2025-11-29T08:43:30.389+08:00 and status AVAILABLE 2025-11-29 08:43:30.394 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-11-29T08:43:30.393+08:00 2025-11-29 08:43:30.394 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/time, params:null 2025-11-29 08:43:30.394 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.s.compute.HostAgentDaoManager : Updated agent 13 last available time 2025-11-29T08:43:30.391+08:00 and status AVAILABLE 2025-11-29 08:43:30.394 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/time, params:null 2025-11-29 08:43:30.395 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.202.22.51, timestamp=2025-11-29T08:43:30.392882173+08:00 2025-11-29 08:43:30.395 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=11, ip=10.202.22.47, timestamp=2025-11-29T08:43:30.394546709+08:00 2025-11-29 08:43:30.396 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-11-29T08:43:30.395+08:00 2025-11-29 08:43:30.396 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=11, lastAvailableTime=2025-11-29T08:43:30.395+08:00 2025-11-29 08:43:30.397 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-11-29T08:43:30.393+08:00 and status AVAILABLE 2025-11-29 08:43:30.398 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.s.compute.HostAgentDaoManager : Updated agent 11 last available time 2025-11-29T08:43:30.395+08:00 and status AVAILABLE 2025-11-29 08:43:30.399 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-11-29T08:43:30.395+08:00 and status AVAILABLE 2025-11-29 08:43:30.416 INFO 12717 --- [host-check-agent-status11,883683064d2ddd8d,44bf519af34e6db7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.48 2025-11-29 08:43:30.416 INFO 12717 --- [host-check-agent-status11,883683064d2ddd8d,44bf519af34e6db7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.48 2025-11-29 08:43:30.416 INFO 12717 --- [host-check-agent-status11,883683064d2ddd8d,44bf519af34e6db7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.48 2025-11-29 08:43:30.582 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=10, ip=10.202.22.61, timestamp=2025-11-29T08:43:30.580006250+08:00 2025-11-29 08:43:30.583 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=10, lastAvailableTime=2025-11-29T08:43:30.582+08:00 2025-11-29 08:43:30.585 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2025-11-29T08:43:30.582+08:00 and status AVAILABLE 2025-11-29 08:43:30.589 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=9, ip=10.202.22.60, timestamp=2025-11-29T08:43:30.583251743+08:00 2025-11-29 08:43:30.589 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=9, lastAvailableTime=2025-11-29T08:43:30.589+08:00 2025-11-29 08:43:30.592 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.s.compute.HostAgentDaoManager : Updated agent 10 last available time 2025-11-29T08:43:30.589+08:00 and status AVAILABLE 2025-11-29 08:43:30.603 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-11-29 08:43:30.610 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=9 2025-11-29 08:43:30.610 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-11-29 08:43:30.610 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=10 2025-11-29 08:43:30.610 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-11-29 08:43:30.610 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2025-11-29 08:43:30.610 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=13 2025-11-29 08:43:30.610 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=11 2025-11-29 08:43:30.610 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-11-29 08:43:30.610 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=8 2025-11-29 08:43:30.610 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=12 2025-11-29 08:43:30.613 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-11-29 08:43:30.644 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:43:30.644 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:43:30.644 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:43:30.644 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:43:30.645 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:43:30.645 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:43:30.645 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:43:30.645 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:43:30.646 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:43:30.646 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:43:30.646 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:43:30.646 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:43:30.646 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:43:30.646 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:43:30.646 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:43:30.646 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:43:30.647 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:43:30.647 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:43:30.648 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:43:30.649 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:43:30.649 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:43:30.651 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:43:30.664 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:43:30.665 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:43:30.666 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.668 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.668 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.668 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.669 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.669 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.670 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.671 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.672 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.674 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.676 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.685 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:43:30.723 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-11-29 08:43:30.723 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=16 2025-11-29 08:43:30.723 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=28 2025-11-29 08:43:30.728 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=25 2025-11-29 08:43:30.728 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-11-29 08:43:30.736 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-11-29 08:43:30.737 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-11-29 08:43:30.740 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=34 2025-11-29 08:43:30.742 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-11-29 08:43:30.742 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=31 2025-11-29 08:43:30.745 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=16 2025-11-29 08:43:30.746 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-11-29 08:43:30.747 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=28 2025-11-29 08:43:30.748 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-11-29 08:43:30.748 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=25 2025-11-29 08:43:30.750 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-11-29 08:43:30.752 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-11-29 08:43:30.752 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=34 2025-11-29 08:43:30.754 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-11-29 08:43:30.755 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=31 2025-11-29 08:43:30.775 INFO 12717 --- [prometheus-second-schedule-1,a8408ce097b280e1,9ca0097e4c57dda1] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=42, collectTaskCount=42 2025-11-29 08:43:30.786 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=22 2025-11-29 08:43:30.797 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=22 2025-11-29 08:43:30.904 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=37 2025-11-29 08:43:30.914 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=37 2025-11-29 08:43:30.961 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.968 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.968 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.969 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.969 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.970 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.971 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.971 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.971 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.971 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.974 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:30.974 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:43:31.025 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-11-29 08:43:31.026 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=35 2025-11-29 08:43:31.027 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=26 2025-11-29 08:43:31.029 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-11-29 08:43:31.032 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-11-29 08:43:31.035 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=17 2025-11-29 08:43:31.035 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-11-29 08:43:31.035 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=32 2025-11-29 08:43:31.036 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-11-29 08:43:31.036 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=35 2025-11-29 08:43:31.036 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=26 2025-11-29 08:43:31.038 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-11-29 08:43:31.038 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=29 2025-11-29 08:43:31.041 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-11-29 08:43:31.044 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=17 2025-11-29 08:43:31.044 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=32 2025-11-29 08:43:31.046 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-11-29 08:43:31.047 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=29 2025-11-29 08:43:31.049 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-11-29 08:43:31.056 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.057 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.057 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.058 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.059 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-11-29 08:43:31.060 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.063 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.063 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.066 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.066 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.079 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.106 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-11-29 08:43:31.114 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=33 2025-11-29 08:43:31.116 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-11-29 08:43:31.120 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=18 2025-11-29 08:43:31.121 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-11-29 08:43:31.122 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=36 2025-11-29 08:43:31.123 INFO 12717 --- [host-check-host-status4,0c05aa171b6fe023,9b482b7844a2912f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-11-29 08:43:31.131 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-11-29 08:43:31.133 INFO 12717 --- [host-check-host-status10,0c05aa171b6fe023,78dfce8f6d8a5a92] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=33 2025-11-29 08:43:31.137 INFO 12717 --- [host-check-host-status6,0c05aa171b6fe023,130e86b18fea2205] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-11-29 08:43:31.138 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-11-29 08:43:31.139 INFO 12717 --- [host-check-host-status12,0c05aa171b6fe023,ebbb03e5fced0377] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=36 2025-11-29 08:43:31.139 INFO 12717 --- [host-check-host-status2,0c05aa171b6fe023,84de5798e386554e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=18 2025-11-29 08:43:31.141 INFO 12717 --- [host-check-host-status3,0c05aa171b6fe023,46e33b4de02a673a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-11-29 08:43:31.143 INFO 12717 --- [host-check-host-status1,0c05aa171b6fe023,4697a3685530e4f4] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-11-29 08:43:31.150 INFO 12717 --- [host-check-host-status5,0c05aa171b6fe023,ce243dec68036c92] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-11-29 08:43:31.154 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=23 2025-11-29 08:43:31.164 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=23 2025-11-29 08:43:31.281 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=38 2025-11-29 08:43:31.290 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=38 2025-11-29 08:43:31.298 WARN 12717 --- [host-check-clockdiff10,63a56589a6ba43b2,d2721280d9c28591] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.47], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:31.298 ERROR 12717 --- [host-check-clockdiff10,63a56589a6ba43b2,d2721280d9c28591] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.47] 2025-11-29 08:43:31.298 WARN 12717 --- [host-check-clockdiff10,63a56589a6ba43b2,d2721280d9c28591] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=11, ip=10.202.22.47, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.47 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:31.317 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=27 2025-11-29 08:43:31.322 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-91,51cedc9b0a15682a,478c9fe64046a443] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=51cedc9b0a15682a, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:31.327 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=30 2025-11-29 08:43:31.330 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.330 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:43:31.331 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-91,51cedc9b0a15682a,478c9fe64046a443] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=51cedc9b0a15682a, duration=10 ms] 2025-11-29 08:43:31.331 INFO 12717 --- [host-check-host-status9,0c05aa171b6fe023,8e70e7b800491477] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=27 2025-11-29 08:43:31.337 INFO 12717 --- [host-check-host-status8,0c05aa171b6fe023,fb7ecc2eacd87d99] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=30 2025-11-29 08:43:31.403 INFO 12717 --- [host-check-clockdiff12,63a56589a6ba43b2,3aee8aad60223e89] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764377011 -2 -2 , error= 2025-11-29 08:43:31.500 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=24 2025-11-29 08:43:31.510 INFO 12717 --- [host-check-host-status7,0c05aa171b6fe023,9b9a786cfae0bb7f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=24 2025-11-29 08:43:31.516 WARN 12717 --- [host-check-clockdiff6,63a56589a6ba43b2,479d553a63557e0c] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.54], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:31.516 ERROR 12717 --- [host-check-clockdiff6,63a56589a6ba43b2,479d553a63557e0c] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.54] 2025-11-29 08:43:31.516 WARN 12717 --- [host-check-clockdiff6,63a56589a6ba43b2,479d553a63557e0c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.202.22.54, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.54 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:31.523 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:31.524 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:31.527 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=39 2025-11-29 08:43:31.537 INFO 12717 --- [host-check-host-status11,0c05aa171b6fe023,1707107d7a6f26e3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=39 2025-11-29 08:43:31.633 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-87,c5f00af164b02a90,dfdb70241a6f0755] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=c5f00af164b02a90, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:31.633 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-87,c5f00af164b02a90,dfdb70241a6f0755] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=c5f00af164b02a90, duration=1 ms] 2025-11-29 08:43:31.695 INFO 12717 --- [ocp-async-33,8836d8cb9aa0726d,443bea2fd5903562] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:31.900 WARN 12717 --- [host-check-clockdiff5,63a56589a6ba43b2,9c3090e743fc6efa] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.53], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:31.900 ERROR 12717 --- [host-check-clockdiff5,63a56589a6ba43b2,9c3090e743fc6efa] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.53] 2025-11-29 08:43:31.900 WARN 12717 --- [host-check-clockdiff5,63a56589a6ba43b2,9c3090e743fc6efa] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.202.22.53, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.53 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:32.407 INFO 12717 --- [prometheus-minute-schedule-1,e6dbfb9bb97761ae,f1bdb6a8896e3228] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:32.414 INFO 12717 --- [prometheus-minute-schedule-1,e6dbfb9bb97761ae,f1bdb6a8896e3228] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:32.533 WARN 12717 --- [host-check-clockdiff2,63a56589a6ba43b2,e693527531dc81f5] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.50], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:32.533 ERROR 12717 --- [host-check-clockdiff2,63a56589a6ba43b2,e693527531dc81f5] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.50] 2025-11-29 08:43:32.533 WARN 12717 --- [host-check-clockdiff2,63a56589a6ba43b2,e693527531dc81f5] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.202.22.50, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.50 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:32.698 WARN 12717 --- [host-check-clockdiff3,63a56589a6ba43b2,d0ab82b80f473517] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.51], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:32.698 ERROR 12717 --- [host-check-clockdiff3,63a56589a6ba43b2,d0ab82b80f473517] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.51] 2025-11-29 08:43:32.698 WARN 12717 --- [host-check-clockdiff3,63a56589a6ba43b2,d0ab82b80f473517] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.202.22.51, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.51 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:32.813 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,46213753773d7649,d821ec07cbce9776] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=46213753773d7649, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:32.813 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,46213753773d7649,d821ec07cbce9776] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=46213753773d7649, duration=0 ms] 2025-11-29 08:43:32.852 INFO 12717 --- [ocp-schedules-29,34e24922e230e149,7b3819c80a78deef] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:43:32.873 WARN 12717 --- [scanner-11213814483-1,34e24922e230e149,979f7f28a35921e6] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:43:32.894 WARN 12717 --- [host-check-clockdiff9,63a56589a6ba43b2,80ecb236d418dacf] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.61], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:32.894 ERROR 12717 --- [host-check-clockdiff9,63a56589a6ba43b2,80ecb236d418dacf] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.61] 2025-11-29 08:43:32.894 WARN 12717 --- [host-check-clockdiff9,63a56589a6ba43b2,80ecb236d418dacf] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=10, ip=10.202.22.61, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.61 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:32.957 WARN 12717 --- [host-check-clockdiff11,63a56589a6ba43b2,d5ac131940f4f9d5] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.48], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:32.957 ERROR 12717 --- [host-check-clockdiff11,63a56589a6ba43b2,d5ac131940f4f9d5] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.48] 2025-11-29 08:43:32.957 WARN 12717 --- [host-check-clockdiff11,63a56589a6ba43b2,d5ac131940f4f9d5] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=12, ip=10.202.22.48, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.48 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:33.129 INFO 12717 --- [ocp-async-34,b00104c584d0a10a,2c68838d418d5279] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:43:33.250 WARN 12717 --- [host-check-clockdiff8,63a56589a6ba43b2,89eb14c05322b744] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.60], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:33.250 ERROR 12717 --- [host-check-clockdiff8,63a56589a6ba43b2,89eb14c05322b744] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.60] 2025-11-29 08:43:33.250 WARN 12717 --- [host-check-clockdiff8,63a56589a6ba43b2,89eb14c05322b744] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=9, ip=10.202.22.60, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.60 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:33.385 WARN 12717 --- [host-check-clockdiff4,63a56589a6ba43b2,e089badbeecfbf09] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.52], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:33.385 ERROR 12717 --- [host-check-clockdiff4,63a56589a6ba43b2,e089badbeecfbf09] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.52] 2025-11-29 08:43:33.385 WARN 12717 --- [host-check-clockdiff4,63a56589a6ba43b2,e089badbeecfbf09] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.202.22.52, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.52 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:43:33.695 INFO 12717 --- [ocp-async-37,7b6fdb532945664e,4bed2532c606ac7a] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:33.930 INFO 12717 --- [ocp-schedules-3,b6f8faa9cdcada1e,c9a66b18280e8ae4] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:43:33.930 INFO 12717 --- [ocp-schedules-3,b6f8faa9cdcada1e,c9a66b18280e8ae4] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=9 2025-11-29 08:43:33.930 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2025-11-29 08:43:33.930 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=6 2025-11-29 08:43:33.930 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2025-11-29 08:43:33.930 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2025-11-29 08:43:33.930 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2025-11-29 08:43:33.930 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2025-11-29 08:43:33.930 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=8 2025-11-29 08:43:33.930 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=10 2025-11-29 08:43:33.930 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=11 2025-11-29 08:43:33.930 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=12 2025-11-29 08:43:33.931 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=13 2025-11-29 08:43:33.937 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:43:33.938 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:43:33.938 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:43:33.938 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:43:33.938 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:43:33.938 INFO 12717 --- [ocp-schedules-3,b6f8faa9cdcada1e,c9a66b18280e8ae4] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:43:33.938 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:43:33.938 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:43:33.938 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:43:33.939 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:43:33.940 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:43:33.940 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:43:33.940 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:43:33.940 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:43:33.940 INFO 12717 --- [ocp-schedules-3,b6f8faa9cdcada1e,c9a66b18280e8ae4] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:43:33.940 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:43:33.940 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:43:33.940 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:43:33.941 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:43:33.941 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:43:33.941 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:43:33.941 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:43:33.942 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:43:33.943 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:43:33.970 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.970 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.970 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.970 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.971 INFO 12717 --- [ocp-schedules-3,b6f8faa9cdcada1e,c9a66b18280e8ae4] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.971 INFO 12717 --- [ForkJoinPool.commonPool-worker-26,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.971 INFO 12717 --- [ForkJoinPool.commonPool-worker-23,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.971 INFO 12717 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.972 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.973 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.974 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:33.974 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62889/api/v1/log/alarms, params:null 2025-11-29 08:43:34.176 INFO 12717 --- [ocp-schedules-3,b6f8faa9cdcada1e,c9a66b18280e8ae4] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms finished. 2025-11-29 08:43:34.524 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:34.528 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:34.725 INFO 12717 --- [ocp-schedules-25,68872e369144c252,f3941c0f0550d846] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all tenant information executed completed, allCount=2, successCount=2 2025-11-29 08:43:35.407 INFO 12717 --- [prometheus-minute-schedule-1,dd59949c0a2764c4,7dffbf38e5a37da2] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:35.416 INFO 12717 --- [prometheus-minute-schedule-1,dd59949c0a2764c4,7dffbf38e5a37da2] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:35.571 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,61140c11ba2cfad1,45babec3dd7a52ef] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=61140c11ba2cfad1, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:35.577 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,61140c11ba2cfad1,45babec3dd7a52ef] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=61140c11ba2cfad1, duration=7 ms] 2025-11-29 08:43:35.695 INFO 12717 --- [ocp-async-37,939bdb92773b1131,601d3e5a2e1caef3] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:35.740 INFO 12717 --- [ocp-schedules-7,e9a337bc870e76df,10b5210e2c98e38a] c.o.o.s.c.scheduled.ClusterSchedules : Sync root server list. 2025-11-29 08:43:35.742 INFO 12717 --- [ocp-schedules-7,e9a337bc870e76df,10b5210e2c98e38a] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:43:35.766 INFO 12717 --- [host-check-clockdiff7,63a56589a6ba43b2,5bbd9b0684f2485b] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764377015 0 0 , error= 2025-11-29 08:43:37.249 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,fb06871be347bb02,1a4feaf015d8ee4b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=fb06871be347bb02, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:37.255 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,fb06871be347bb02,1a4feaf015d8ee4b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=fb06871be347bb02, duration=7 ms] 2025-11-29 08:43:37.528 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:37.530 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:37.698 INFO 12717 --- [ocp-async-35,646a38e71c57f244,cad7f15c6bffe3a9] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:38.191 INFO 12717 --- [ocp-schedules-10,0c788f0c571c95b8,c4eb2ab4b18804b1] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:43:38.194 INFO 12717 --- [ocp-schedules-10,0c788f0c571c95b8,c4eb2ab4b18804b1] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:43:38.407 INFO 12717 --- [prometheus-minute-schedule-1,a3eba09971205961,159aa1ff1e3f89d2] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:38.414 INFO 12717 --- [prometheus-minute-schedule-1,a3eba09971205961,159aa1ff1e3f89d2] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:39.054 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,a677bfeb44c06444,9ef3d4f84f7294ce] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=a677bfeb44c06444, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:39.054 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,a677bfeb44c06444,9ef3d4f84f7294ce] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=a677bfeb44c06444, duration=1 ms] 2025-11-29 08:43:39.663 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.o.s.c.scheduled.TenantSchedules : Check suspend tenant compaction status 2025-11-29 08:43:39.663 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] s.t.b.t.CheckSuspendTenantCompactionTask : Check tenant compaction status. 2025-11-29 08:43:39.681 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:43:39.681061+08:00, lastProcessTime=2025-11-29T08:43:29.682058+08:00 2025-11-29 08:43:39.695 INFO 12717 --- [ocp-async-35,3f8729053fad472e,6b254f3d94292a96] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:39.762 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:39.763 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-11-29 08:43:39.791 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:43:39.791 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:43:39.791 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:43:39.791 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:43:39.795 INFO 12717 --- [ocp-schedules-6,11d788f9a6f19408,f64211c397666d6d] c.o.o.s.common.scheduled.PerfSchedules : Collect slow sql for alarm, time=2025-11-29T08:43:39.795+08:00, clusters=[1, 2] 2025-11-29 08:43:39.803 INFO 12717 --- [ocp-schedules-6,11d788f9a6f19408,f64211c397666d6d] c.o.o.p.s.i.alarm.SlowSqlDataSource : query sqlCount of obCluster ocpserver for slowSqlStat, sql count is 0. 2025-11-29 08:43:39.805 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:43:39.805 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:43:39.805 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:43:39.805 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:43:39.806 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:39.807 INFO 12717 --- [ocp-schedules-21,dc493ef143361fea,7c5015b0d6f8060a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-11-29 08:43:39.930 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,9c8cc11c878d2e75,fef1562c143cca8f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=9c8cc11c878d2e75, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:39.930 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,9c8cc11c878d2e75,fef1562c143cca8f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=9c8cc11c878d2e75, duration=1 ms] 2025-11-29 08:43:40.011 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:43:39.681061+08:00 2025-11-29 08:43:40.017 INFO 12717 --- [ocp-schedules-15,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=3 2025-11-29 08:43:40.020 INFO 12717 --- [ocp-schedules-23,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=3 2025-11-29 08:43:40.060 INFO 12717 --- [ocp-schedules-6,11d788f9a6f19408,f64211c397666d6d] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for DB, save 15 to SqlAggregateData 2025-11-29 08:43:40.081 INFO 12717 --- [Check obproxy2,f6cdaabb98f69ed9,5bb8be841e8f264d] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:43:40.081 INFO 12717 --- [Check obproxy1,4d1fe94252553905,9e99aa68b512608e] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:43:40.081 INFO 12717 --- [Check obproxy1,4d1fe94252553905,9e99aa68b512608e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:40.081 INFO 12717 --- [Check obproxy2,f6cdaabb98f69ed9,5bb8be841e8f264d] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:40.081 INFO 12717 --- [Check obproxy3,9cfe921a35624421,bc44bb706671fe4a] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:43:40.082 INFO 12717 --- [Check obproxy3,9cfe921a35624421,bc44bb706671fe4a] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:40.085 INFO 12717 --- [Collect obproxy3,85efeaa854f5b9b7,c51b8be2c08af147] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:43:40.085 INFO 12717 --- [Collect obproxy3,85efeaa854f5b9b7,c51b8be2c08af147] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:40.089 INFO 12717 --- [Collect obproxy1,b995aa9144785669,e5ece0edcad3274f] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:43:40.089 INFO 12717 --- [Collect obproxy1,b995aa9144785669,e5ece0edcad3274f] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:40.092 INFO 12717 --- [Collect obproxy2,9169623359d7c4db,49e19f763c996c26] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:43:40.092 INFO 12717 --- [Collect obproxy2,9169623359d7c4db,49e19f763c996c26] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:43:40.123 INFO 12717 --- [Check obproxy2,f6cdaabb98f69ed9,5bb8be841e8f264d] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:43:40.128 INFO 12717 --- [Check obproxy3,9cfe921a35624421,bc44bb706671fe4a] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:43:40.128 INFO 12717 --- [Collect obproxy3,85efeaa854f5b9b7,c51b8be2c08af147] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:43:40.129 INFO 12717 --- [Check obproxy2,f6cdaabb98f69ed9,5bb8be841e8f264d] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-11-29 08:43:40.129 INFO 12717 --- [Check obproxy2,f6cdaabb98f69ed9,5bb8be841e8f264d] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 2, need to sync to meta: false 2025-11-29 08:43:40.132 INFO 12717 --- [Check obproxy3,9cfe921a35624421,bc44bb706671fe4a] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2025-11-29 08:43:40.132 INFO 12717 --- [Check obproxy3,9cfe921a35624421,bc44bb706671fe4a] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 3 proxy_id: 3, need to sync to meta: false 2025-11-29 08:43:40.133 INFO 12717 --- [Collect obproxy3,85efeaa854f5b9b7,c51b8be2c08af147] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:43:40.134 INFO 12717 --- [Check obproxy1,4d1fe94252553905,9e99aa68b512608e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:43:40.134 INFO 12717 --- [Collect obproxy1,b995aa9144785669,e5ece0edcad3274f] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:43:40.137 INFO 12717 --- [Collect obproxy3,85efeaa854f5b9b7,c51b8be2c08af147] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:43:40.137 INFO 12717 --- [Collect obproxy1,b995aa9144785669,e5ece0edcad3274f] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:43:40.137 INFO 12717 --- [Collect obproxy2,9169623359d7c4db,49e19f763c996c26] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:43:40.139 INFO 12717 --- [Check obproxy1,4d1fe94252553905,9e99aa68b512608e] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-11-29 08:43:40.139 INFO 12717 --- [Check obproxy1,4d1fe94252553905,9e99aa68b512608e] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 1, need to sync to meta: false 2025-11-29 08:43:40.140 INFO 12717 --- [Collect obproxy1,b995aa9144785669,e5ece0edcad3274f] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:43:40.143 INFO 12717 --- [Collect obproxy2,9169623359d7c4db,49e19f763c996c26] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:43:40.147 INFO 12717 --- [Collect obproxy2,9169623359d7c4db,49e19f763c996c26] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:43:40.221 INFO 12717 --- [ocp-schedules-20,516da3556bf2b8c9,8efca48c65eb07dc] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:43:40.275 INFO 12717 --- [ocp-schedules-11,ab98e978e0619e5e,31053f3f80d0febc] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=2, successCount=2 2025-11-29 08:43:40.321 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,5ffd46fc43e061fc,2f5fff408f92ed02] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=5ffd46fc43e061fc, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:40.322 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,5ffd46fc43e061fc,2f5fff408f92ed02] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=5ffd46fc43e061fc, duration=1 ms] 2025-11-29 08:43:40.381 INFO 12717 --- [ocp-schedules-27,f01a2e37dd41934c,ad94f46d40a1135d] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:43:40.530 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:40.532 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:41.408 INFO 12717 --- [prometheus-minute-schedule-1,87203018324e6492,b98521ea43e7ae0b] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:41.435 INFO 12717 --- [prometheus-minute-schedule-1,87203018324e6492,b98521ea43e7ae0b] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:41.696 INFO 12717 --- [ocp-async-34,7a09043df8ec24a9,c472dafb06bd0367] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:42.698 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,e89cd446ad57adff,501096f740eee3db] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=e89cd446ad57adff, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:42.708 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,e89cd446ad57adff,501096f740eee3db] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=e89cd446ad57adff, duration=17 ms] 2025-11-29 08:43:43.017 INFO 12717 --- [ocp-schedules-15,be80173dd4326a26,58a2fe4ecd4f73b9] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:43:43.041 INFO 12717 --- [alarm-detect-5,be80173dd4326a26,ccb47de858cd14e3] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008327, data is empty. 2025-11-29 08:43:43.042 INFO 12717 --- [alarm-detect-6,be80173dd4326a26,3754570c82d1a229] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008327, data is empty. 2025-11-29 08:43:43.044 INFO 12717 --- [alarm-detect-4,be80173dd4326a26,e93d75d05d93872f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008327, data is empty. 2025-11-29 08:43:43.050 INFO 12717 --- [alarm-detect-5,be80173dd4326a26,ccb47de858cd14e3] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008342, data is empty. 2025-11-29 08:43:43.052 INFO 12717 --- [alarm-detect-6,be80173dd4326a26,3754570c82d1a229] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008342, data is empty. 2025-11-29 08:43:43.054 INFO 12717 --- [alarm-detect-4,be80173dd4326a26,e93d75d05d93872f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008342, data is empty. 2025-11-29 08:43:43.061 WARN 12717 --- [scanner-11213814711-1,be80173dd4326a26,2b92f1a409f6dc8f] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:43:43.131 INFO 12717 --- [ocp-async-21,31472868b05776d5,cf3dea143b38d5f9] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:43:43.154 INFO 12717 --- [alarm-detect-20,be80173dd4326a26,13830f70fc686eb9] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008327, data is empty. 2025-11-29 08:43:43.162 INFO 12717 --- [alarm-detect-20,be80173dd4326a26,13830f70fc686eb9] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008342, data is empty. 2025-11-29 08:43:43.533 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:43.534 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:43.696 INFO 12717 --- [ocp-async-17,25e6a13b3514172a,f63731142ab9e1f0] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:44.407 INFO 12717 --- [prometheus-minute-schedule-1,ac0be22dc82272ba,96dd0ea0dbdfc182] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:44.423 INFO 12717 --- [prometheus-minute-schedule-1,ac0be22dc82272ba,96dd0ea0dbdfc182] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:45.643 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,51e8d4d72465429b,cab1261d3856043d] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=51e8d4d72465429b, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:45.652 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,51e8d4d72465429b,cab1261d3856043d] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=51e8d4d72465429b, duration=10 ms] 2025-11-29 08:43:45.697 INFO 12717 --- [ocp-async-8,af316be76577b456,b366f06686baa383] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:45.829 INFO 12717 --- [ocp-schedules-7,e9a337bc870e76df,10b5210e2c98e38a] .o.o.s.t.b.c.AbstractShardingClusterTask : Refresh root service list executed completed, allCount=2, successCount=2 2025-11-29 08:43:46.534 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:46.535 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:47.020 INFO 12717 --- [ocp-schedules-16,fb7dbb3edcf7f099,8515fc1661b1d278] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-11-29 08:43:47.120 INFO 12717 --- [ocp-schedules-16,fb7dbb3edcf7f099,8515fc1661b1d278] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:43:47.120 INFO 12717 --- [ocp-schedules-16,fb7dbb3edcf7f099,8515fc1661b1d278] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:43:47.306 INFO 12717 --- [ocp-schedules-7,29b3c0d74ba9b60a,6f7e5fe82175cd3f] c.o.o.s.c.scheduled.TenantSchedules : Collect all tenant major compaction 2025-11-29 08:43:47.306 INFO 12717 --- [ocp-schedules-7,29b3c0d74ba9b60a,6f7e5fe82175cd3f] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2025-11-29 08:43:47.392 INFO 12717 --- [Collect all tenants major compactions2,29b3c0d74ba9b60a,577fbe9a663423ef] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:ocp_monitor 2025-11-29 08:43:47.393 INFO 12717 --- [Collect all tenants major compactions3,29b3c0d74ba9b60a,15b3527c24935e61] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:sys 2025-11-29 08:43:47.393 INFO 12717 --- [Collect all tenants major compactions1,29b3c0d74ba9b60a,1bb98ef4b4729296] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:ocp_meta 2025-11-29 08:43:47.396 INFO 12717 --- [Collect all tenants major compactions7,29b3c0d74ba9b60a,a34568497c782001] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:pipeline 2025-11-29 08:43:47.397 INFO 12717 --- [Collect all tenants major compactions9,29b3c0d74ba9b60a,32a0eb817d8c8f90] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:wecom_bot 2025-11-29 08:43:47.397 INFO 12717 --- [Collect all tenants major compactions5,29b3c0d74ba9b60a,869114f812f7119c] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:nextcloud 2025-11-29 08:43:47.398 INFO 12717 --- [Collect all tenants major compactions6,29b3c0d74ba9b60a,52bf2c9f64e21745] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:oms 2025-11-29 08:43:47.398 INFO 12717 --- [Collect all tenants major compactions4,29b3c0d74ba9b60a,118cacdad717818e] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:lhp6796 2025-11-29 08:43:47.405 INFO 12717 --- [Collect all tenants major compactions8,29b3c0d74ba9b60a,bc425e4d8f4e3e96] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:sys 2025-11-29 08:43:47.406 INFO 12717 --- [Collect all tenants major compactions10,29b3c0d74ba9b60a,733f95ce325cfd86] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:zto 2025-11-29 08:43:47.407 INFO 12717 --- [prometheus-minute-schedule-1,4b0976e41d50df58,a338334fda52394e] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:47.428 INFO 12717 --- [prometheus-minute-schedule-1,4b0976e41d50df58,a338334fda52394e] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:47.461 INFO 12717 --- [Collect all tenants major compactions10,29b3c0d74ba9b60a,733f95ce325cfd86] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:47.461 INFO 12717 --- [Collect all tenants major compactions2,29b3c0d74ba9b60a,577fbe9a663423ef] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:47.461 INFO 12717 --- [Collect all tenants major compactions5,29b3c0d74ba9b60a,869114f812f7119c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:47.461 INFO 12717 --- [Collect all tenants major compactions6,29b3c0d74ba9b60a,52bf2c9f64e21745] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:47.461 INFO 12717 --- [Collect all tenants major compactions4,29b3c0d74ba9b60a,118cacdad717818e] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:47.461 INFO 12717 --- [Collect all tenants major compactions8,29b3c0d74ba9b60a,bc425e4d8f4e3e96] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:47.462 INFO 12717 --- [Collect all tenants major compactions3,29b3c0d74ba9b60a,15b3527c24935e61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:47.462 INFO 12717 --- [Collect all tenants major compactions1,29b3c0d74ba9b60a,1bb98ef4b4729296] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:47.463 INFO 12717 --- [Collect all tenants major compactions5,29b3c0d74ba9b60a,869114f812f7119c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1018] 2025-11-29 08:43:47.463 INFO 12717 --- [Collect all tenants major compactions10,29b3c0d74ba9b60a,733f95ce325cfd86] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1022] 2025-11-29 08:43:47.463 INFO 12717 --- [Collect all tenants major compactions4,29b3c0d74ba9b60a,118cacdad717818e] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1010] 2025-11-29 08:43:47.464 INFO 12717 --- [Collect all tenants major compactions6,29b3c0d74ba9b60a,52bf2c9f64e21745] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1014] 2025-11-29 08:43:47.464 INFO 12717 --- [Collect all tenants major compactions8,29b3c0d74ba9b60a,bc425e4d8f4e3e96] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-11-29 08:43:47.464 INFO 12717 --- [Collect all tenants major compactions3,29b3c0d74ba9b60a,15b3527c24935e61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-11-29 08:43:47.465 INFO 12717 --- [Collect all tenants major compactions2,29b3c0d74ba9b60a,577fbe9a663423ef] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1004] 2025-11-29 08:43:47.465 INFO 12717 --- [Collect all tenants major compactions1,29b3c0d74ba9b60a,1bb98ef4b4729296] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2025-11-29 08:43:47.468 INFO 12717 --- [Collect all tenants major compactions7,29b3c0d74ba9b60a,a34568497c782001] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:47.470 INFO 12717 --- [Collect all tenants major compactions7,29b3c0d74ba9b60a,a34568497c782001] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1020] 2025-11-29 08:43:47.473 INFO 12717 --- [Collect all tenants major compactions1,29b3c0d74ba9b60a,1bb98ef4b4729296] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:ocp_meta 2025-11-29 08:43:47.473 INFO 12717 --- [Collect all tenants major compactions8,29b3c0d74ba9b60a,bc425e4d8f4e3e96] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:sys 2025-11-29 08:43:47.473 INFO 12717 --- [Collect all tenants major compactions5,29b3c0d74ba9b60a,869114f812f7119c] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:nextcloud 2025-11-29 08:43:47.473 INFO 12717 --- [Collect all tenants major compactions2,29b3c0d74ba9b60a,577fbe9a663423ef] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:ocp_monitor 2025-11-29 08:43:47.473 INFO 12717 --- [Collect all tenants major compactions6,29b3c0d74ba9b60a,52bf2c9f64e21745] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:oms 2025-11-29 08:43:47.473 INFO 12717 --- [Collect all tenants major compactions3,29b3c0d74ba9b60a,15b3527c24935e61] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:sys 2025-11-29 08:43:47.473 INFO 12717 --- [Collect all tenants major compactions10,29b3c0d74ba9b60a,733f95ce325cfd86] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:zto 2025-11-29 08:43:47.473 INFO 12717 --- [Collect all tenants major compactions4,29b3c0d74ba9b60a,118cacdad717818e] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:lhp6796 2025-11-29 08:43:47.474 INFO 12717 --- [Collect all tenants major compactions7,29b3c0d74ba9b60a,a34568497c782001] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:pipeline 2025-11-29 08:43:47.476 INFO 12717 --- [Collect all tenants major compactions9,29b3c0d74ba9b60a,32a0eb817d8c8f90] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:47.478 INFO 12717 --- [Collect all tenants major compactions9,29b3c0d74ba9b60a,32a0eb817d8c8f90] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1024] 2025-11-29 08:43:47.485 INFO 12717 --- [Collect all tenants major compactions9,29b3c0d74ba9b60a,32a0eb817d8c8f90] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:wecom_bot 2025-11-29 08:43:47.487 INFO 12717 --- [ocp-schedules-7,29b3c0d74ba9b60a,6f7e5fe82175cd3f] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2025-11-29 08:43:47.697 INFO 12717 --- [ocp-async-26,43624a2e6ec26fd2,0f1b34cfc03268ee] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:47.830 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,5ee52ff9c2cded1e,6217767193714035] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=5ee52ff9c2cded1e, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:47.831 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,5ee52ff9c2cded1e,6217767193714035] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=5ee52ff9c2cded1e, duration=3 ms] 2025-11-29 08:43:47.869 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,95a88f368da2b860,eb0733c762e5c528] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=95a88f368da2b860, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:47.869 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,95a88f368da2b860,eb0733c762e5c528] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=95a88f368da2b860, duration=1 ms] 2025-11-29 08:43:48.097 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,d95cb6d5f9addfdc,742eb67c29e2a065] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=d95cb6d5f9addfdc, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:48.099 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,d95cb6d5f9addfdc,742eb67c29e2a065] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=d95cb6d5f9addfdc, duration=4 ms] 2025-11-29 08:43:48.224 INFO 12717 --- [ocp-schedules-22,3a34c7b80272a791,df3a1e7fe147bfd7] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:43:48.226 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,eb1b68a099f72edf,9e237d44f7166d54] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=eb1b68a099f72edf, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:48.227 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,eb1b68a099f72edf,9e237d44f7166d54] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=eb1b68a099f72edf, duration=2 ms] 2025-11-29 08:43:48.229 INFO 12717 --- [ocp-schedules-22,3a34c7b80272a791,df3a1e7fe147bfd7] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:43:48.924 INFO 12717 --- [ocp-schedules-17,2ba4cbe532898a77,e9941e6dbfbb35a6] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-11-29 08:43:49.408 INFO 12717 --- [prometheus-cache-monitor1,2138afcf1b7f3d7f,221794042d34f522] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":40355,"minuteWriteQueue":0,"minuteReadCacheSize":17215,"secondWriteCacheSize":10087,"seriesIdKeySize":17387,"secondWriteQueue":0,"minuteWriteCacheSize":17216} 2025-11-29 08:43:49.408 INFO 12717 --- [prometheus-cache-monitor1,2138afcf1b7f3d7f,221794042d34f522] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{ob_sysstat_40012=49, ob_server_resource_memory_assigned_bytes=23, ob_sysstat_40011=49, ob_sysstat_40010=49, ob_unit_num=23, ob_database_disk_required_size=74, go_memstats_mspan_sys_bytes=25, ob_sysstat_30213=42, go_memstats_stack_sys_bytes=25, ob_sysstat_30212=42, ob_sysstat_30217=42, ob_sysstat_30216=42, ob_tenant_request_queue_total_size=72, jvm_gc_memory_allocated_bytes_total=2, ob_system_event_total_waits=59, ob_tenant_disk_v4_log_disk_size=40, ob_sysstat_40019=49, ob_sysstat_40018=49, processor_process_metrics_total=30, jvm_gc_max_data_size_bytes=2, go_memstats_frees_total=25, host_available=29, ob_sysstat_30220=42, ob_sysstat_30222=42, ob_sysstat_30221=42, mysql_output_write_sql_failed_count=29, ob_database_disk_data_size=73, ob_tenant_memstore_freeze_cnt=40, ob_tenant_unit_min_memory=96, ob_sysstat_60087=49, ob_memstore_total_bytes=72, jvm_threads_states_threads=8, process_resident_memory_bytes=25, ob_cache_size_bytes=67, ob_tenant_unit_max_memory=96, input_collect_metrics_total=34, ob_sysstat_50039=50, ob_sysstat_60083=50, ob_sysstat_50038=49, node_disk_io_time_weighted_seconds_total=39, process_open_fds=25, ob_sysstat_50037=49, process_memory_rss_bytes=41, ob_server_resource_cpu_assigned_percent=23, ob_sysstat_50030=50, sql_audit_input_collect_lost_total=36, go_memstats_heap_inuse_bytes=25, ob_sysstat_40118=49, ob_sysstat_40117=49, node_memory_memfree_bytes=32, ob_sysstat_40116=49, ob_sysstat_40005=49, process_exists=50, ob_sysstat_40004=49, ob_sysstat_40003=49, ob_sysstat_40002=49, go_gc_duration_seconds=30, ob_server_resource_disk_bytes=23, ob_sysstat_40001=49, ob_sysstat_40000=49, mysql_output_write_sql_time=33, ob_sysstat_30200=42, odp_current_session=36, ob_sysstat_30201=42, ob_sysstat_30204=42, ob_sysstat_30205=42, ob_sysstat_30208=42, process_thread_count=41, ob_sysstat_30209=42, ob_sysstat_50040=50, ob_sysstat_40009=49, ob_disk_free_bytes=23, ob_sysstat_40008=49, node_filesystem_avail_bytes=67, ob_sysstat_40007=49, ob_sysstat_40006=49, node_time_seconds=31, ob_unit_config_max_i 2025-11-29 08:43:49.535 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:49.538 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:49.697 INFO 12717 --- [ocp-async-28,775fff6eeda03659,a7333ce2170f256b] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:49.709 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:43:49.708229+08:00, lastProcessTime=2025-11-29T08:43:39.681061+08:00 2025-11-29 08:43:50.079 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:43:49.708229+08:00 2025-11-29 08:43:50.228 INFO 12717 --- [ocp-schedules-26,17ee91518e8c4e95,e38645c5336279d9] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:43:50.407 INFO 12717 --- [prometheus-minute-schedule-1,9ca7c1b54e70fff5,7d3fd27d9a205154] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:50.419 INFO 12717 --- [prometheus-minute-schedule-1,9ca7c1b54e70fff5,7d3fd27d9a205154] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:51.698 INFO 12717 --- [ocp-async-25,4fe89f79c97e447a,3be4b97c3f182b0e] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:52.539 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:52.541 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:53.131 INFO 12717 --- [ocp-async-26,f37f5dcf95267100,b660834f9b54d1ac] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:43:53.216 INFO 12717 --- [ocp-schedules-19,ab019880e8eb6219,bff69c383aabc3f5] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:43:53.246 WARN 12717 --- [scanner-11213814915-1,ab019880e8eb6219,b289719a912aa8ab] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:43:53.246 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,98daf4964e2fb28a,0b63f825f8812dd4] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=98daf4964e2fb28a, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:53.253 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,98daf4964e2fb28a,0b63f825f8812dd4] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=98daf4964e2fb28a, duration=9 ms] 2025-11-29 08:43:53.407 INFO 12717 --- [prometheus-minute-schedule-1,e4baf664de6e2080,72d802634bc102ad] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:53.418 INFO 12717 --- [prometheus-minute-schedule-1,e4baf664de6e2080,72d802634bc102ad] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:53.697 INFO 12717 --- [ocp-async-6,95787a13a0622c57,23a7977ec363d2b0] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:54.180 INFO 12717 --- [ocp-schedules-1,9d07c2d9474a0644,1d9e19ac729d7fa3] c.o.o.alarm.service.OcpAlarmServiceImpl : fix alarm target, alarms size 48 2025-11-29 08:43:55.185 INFO 12717 --- [ocp-schedules-3,7beb3391f2d6fa45,9808927a3f325905] c.o.o.s.c.scheduled.ClusterSchedules : Collect all cluster compaction info. 2025-11-29 08:43:55.187 INFO 12717 --- [ocp-schedules-3,7beb3391f2d6fa45,9808927a3f325905] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:43:55.188 INFO 12717 --- [Collect compaction for all clusters1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-11-29 08:43:55.188 INFO 12717 --- [Collect compaction for all clusters2,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-11-29 08:43:55.339 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,f2bab5a82c69a028,cf7b813081d67af8] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=f2bab5a82c69a028, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:55.339 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,f2bab5a82c69a028,cf7b813081d67af8] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=f2bab5a82c69a028, duration=1 ms] 2025-11-29 08:43:55.541 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:55.545 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:55.698 INFO 12717 --- [ocp-async-36,3147b8c3dcb0f0d0,920c144f50b5ed89] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:56.407 INFO 12717 --- [prometheus-minute-schedule-1,449c1afc56077d70,b2a66d37e2e596b5] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:56.420 INFO 12717 --- [prometheus-minute-schedule-1,449c1afc56077d70,b2a66d37e2e596b5] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:56.525 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,6f5877a39c43efbc,65a6009025da35ef] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=6f5877a39c43efbc, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:56.526 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,6f5877a39c43efbc,65a6009025da35ef] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=6f5877a39c43efbc, duration=2 ms] 2025-11-29 08:43:56.672 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,a2a6c8ab0b8b3443,a2f2ddfc235a5e9c] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=a2a6c8ab0b8b3443, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:43:56.673 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,a2a6c8ab0b8b3443,a2f2ddfc235a5e9c] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=a2a6c8ab0b8b3443, duration=2 ms] 2025-11-29 08:43:56.929 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,aa89b07a5bd32c1a,09138b82c4ccaf9b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=aa89b07a5bd32c1a, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:56.934 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-93,aa89b07a5bd32c1a,09138b82c4ccaf9b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=aa89b07a5bd32c1a, duration=7 ms] 2025-11-29 08:43:57.168 INFO 12717 --- [ocp-schedules-23,32b47ec0e523770b,6d820bdeae840fce] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2025-11-29 08:43:57.168 INFO 12717 --- [ocp-schedules-23,32b47ec0e523770b,6d820bdeae840fce] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=817192494921281536 2025-11-29 08:43:57.171 INFO 12717 --- [ocp-schedules-23,32b47ec0e523770b,6d820bdeae840fce] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2025-11-29 08:43:57.171 INFO 12717 --- [ocp-schedules-23,32b47ec0e523770b,6d820bdeae840fce] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=851074473877958656 2025-11-29 08:43:57.174 INFO 12717 --- [ocp-schedules-23,32b47ec0e523770b,6d820bdeae840fce] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2025-11-29 08:43:57.174 INFO 12717 --- [ocp-schedules-23,32b47ec0e523770b,6d820bdeae840fce] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=834880106968510464 2025-11-29 08:43:57.698 INFO 12717 --- [ocp-async-17,463c6a7499f48810,94163475b51f3bba] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:58.240 INFO 12717 --- [ocp-schedules-16,1da7fd72fec51c8f,fab4d88a437e16ac] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:43:58.244 INFO 12717 --- [ocp-schedules-16,1da7fd72fec51c8f,fab4d88a437e16ac] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:43:58.545 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:43:58.547 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:43:58.713 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,85360c9adc0c2f56,4cb7ebd0aed53da7] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=85360c9adc0c2f56, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:43:58.715 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,85360c9adc0c2f56,4cb7ebd0aed53da7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=85360c9adc0c2f56, duration=4 ms] 2025-11-29 08:43:59.407 INFO 12717 --- [prometheus-minute-schedule-1,bce270b42acc7208,227bf5ead6a9db0b] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377040, interval=60 2025-11-29 08:43:59.419 INFO 12717 --- [prometheus-minute-schedule-1,bce270b42acc7208,227bf5ead6a9db0b] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:43:59.583 INFO 12717 --- [ocp-schedules-12,e1a133129b9354e2,b5df8aafb4ff921e] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:43:59.584 INFO 12717 --- [ocp-schedules-12,9ae9389fc1278a64,8a5b29c0ee076773] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-11-29 08:43:59.586 INFO 12717 --- [ocp-schedules-12,9ae9389fc1278a64,8a5b29c0ee076773] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:43:59.698 INFO 12717 --- [ocp-async-8,c08040ca72a2f590,ec17a596f7ecd918] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:43:59.700 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:59.700 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:59.700 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:43:59.701 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:43:59.703 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:43:59.702011+08:00, lastProcessTime=2025-11-29T08:43:49.708229+08:00 2025-11-29 08:43:59.762 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:59.763 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:59.764 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:43:59.813 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:59.814 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:59.815 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:43:59.817 WARN 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=OCP 2025-11-29 08:43:59.817 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:59.821 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:43:59.823 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:43:59.841 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:59.841 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:43:59.918 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:43:59.933 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:43:59.934 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:43:59.935 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:43:59.936 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster ocpserver 2025-11-29 08:43:59.979 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2025-11-29 08:43:59.987 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:43:59.994 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:44:00.030 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:00.031 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:00.032 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:44:00.046 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:43:59.702011+08:00 2025-11-29 08:44:00.092 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:00.093 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:00.093 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:44:00.100 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:00.103 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:44:00.111 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:44:00.135 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:00.135 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:00.157 INFO 12717 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:44:00.235 INFO 12717 --- [ocp-schedules-13,16a24be8bcd9af87,3052a1ac09f49f14] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:44:00.266 INFO 12717 --- [ocp-schedules-16,cfc4618dc5542289,9293baadd4d0ff21] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=BackupAlarmCheckSchedule, method=run 2025-11-29 08:44:00.271 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:00.271 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:12+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:00.271 INFO 12717 --- [ocp-schedules-18,fdb44dedbcfea47d,26b7042ccf675006] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-11-29 08:44:00.271 INFO 12717 --- [ocp-schedules-23,2ad21c79e41a5b4d,fcf65e27f0e9fbad] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-11-29 08:44:00.271 INFO 12717 --- [ocp-schedules-13,703313ae15058e73,a88d4624f2703b55] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-11-29 08:44:00.272 INFO 12717 --- [ocp-schedules-15,6299a976ce7db5b9,57e2a06111d06f1f] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2025-11-29 08:44:00.274 INFO 12717 --- [ocp-schedules-23,2ad21c79e41a5b4d,fcf65e27f0e9fbad] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:44:00.276 INFO 12717 --- [ocp-schedules-13,703313ae15058e73,a88d4624f2703b55] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:44:00.293 INFO 12717 --- [ocp-schedules-16,cfc4618dc5542289,9293baadd4d0ff21] c.o.o.b.i.alarm.BackupAlarmService : Backup alarm check schedule. 2025-11-29 08:44:00.302 INFO 12717 --- [ocp-schedules-10,a8bad0b435e2f5d5,a6b4286fbd280b0b] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-11-29 08:44:00.313 INFO 12717 --- [ocp-schedules-15,6299a976ce7db5b9,57e2a06111d06f1f] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2025-11-29 08:44:00.328 INFO 12717 --- [ocp-schedules-26,b0f2e771b9f8ac93,06b0c382552177a4] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:44:00.329 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:44:00.329 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:44:00.329 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:44:00.329 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:44:00.331 INFO 12717 --- [ocp-schedules-21,95eb62dc9b2c413b,7964c701e12c6057] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:44:00.331 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:44:00.331 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:44:00.331 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:44:00.331 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:44:00.335 INFO 12717 --- [host-check-clockdiff1,95eb62dc9b2c413b,b6b1aeeec47c71a7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.58], timeoutMillis=20000 2025-11-29 08:44:00.335 INFO 12717 --- [host-check-clockdiff2,95eb62dc9b2c413b,ceb462550d838613] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.50], timeoutMillis=20000 2025-11-29 08:44:00.335 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:44:00.335 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:44:00.336 INFO 12717 --- [host-check-clockdiff4,95eb62dc9b2c413b,ce11a78b3d0285dd] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.52], timeoutMillis=20000 2025-11-29 08:44:00.336 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:00.337 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:44:00.337 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:44:00.337 INFO 12717 --- [host-check-clockdiff6,95eb62dc9b2c413b,87a7c3bf9e62fc2c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.54], timeoutMillis=20000 2025-11-29 08:44:00.338 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:00.338 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:00.338 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:44:00.338 INFO 12717 --- [host-check-clockdiff3,95eb62dc9b2c413b,147b12f73ac74c74] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.51], timeoutMillis=20000 2025-11-29 08:44:00.339 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:44:00.339 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:44:00.339 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:44:00.340 INFO 12717 --- [ocp-schedules-24,674148b9db078460,b9686c3b3b548755] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:44:00.340 INFO 12717 --- [host-check-clockdiff10,95eb62dc9b2c413b,f892e6e783615d13] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.47], timeoutMillis=20000 2025-11-29 08:44:00.341 INFO 12717 --- [host-check-clockdiff7,95eb62dc9b2c413b,a38482e3e6b8541e] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.59], timeoutMillis=20000 2025-11-29 08:44:00.341 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:44:00.342 INFO 12717 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.8 2025-11-29 08:44:00.342 INFO 12717 --- [host-check-clockdiff11,95eb62dc9b2c413b,927406fe65369564] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.48], timeoutMillis=20000 2025-11-29 08:44:00.342 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:44:00.342 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:44:00.342 INFO 12717 --- [host-check-agent-status2,674148b9db078460,4b6f515eaa6a1faf] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:44:00.342 INFO 12717 --- [host-check-agent-status1,674148b9db078460,5fe0906d5884044b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:44:00.343 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:44:00.344 INFO 12717 --- [host-check-clockdiff8,95eb62dc9b2c413b,03be3893433b22db] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.60], timeoutMillis=20000 2025-11-29 08:44:00.344 INFO 12717 --- [host-check-clockdiff9,95eb62dc9b2c413b,8c8172af3aafe5a7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.61], timeoutMillis=20000 2025-11-29 08:44:00.344 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:44:00.345 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:44:00.345 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:44:00.345 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:44:00.349 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:44:00.351 INFO 12717 --- [host-check-clockdiff5,95eb62dc9b2c413b,7997d287f671b036] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.53], timeoutMillis=20000 2025-11-29 08:44:00.352 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:44:00.352 INFO 12717 --- [host-check-agent-status6,674148b9db078460,f6d90b29a060aace] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:44:00.352 INFO 12717 --- [host-check-agent-status7,674148b9db078460,a013e607ad4d5d10] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:44:00.352 INFO 12717 --- [host-check-clockdiff12,95eb62dc9b2c413b,f77bc897dda4fef1] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.49], timeoutMillis=20000 2025-11-29 08:44:00.353 INFO 12717 --- [host-check-agent-status8,674148b9db078460,476e74fc63118162] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:44:00.354 INFO 12717 --- [host-check-agent-status9,674148b9db078460,6641f3dadf66379f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:44:00.355 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:44:00.355 INFO 12717 --- [host-check-agent-status4,674148b9db078460,36ff88efcc0c6d2a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:44:00.355 INFO 12717 --- [host-check-agent-status5,674148b9db078460,2dae2df8e8baaa26] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:44:00.355 INFO 12717 --- [host-check-agent-status10,674148b9db078460,819ad72782bb2ae2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:44:00.355 INFO 12717 --- [host-check-agent-status3,674148b9db078460,9c2e51de8e10c89f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:44:00.357 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:44:00.357 INFO 12717 --- [host-check-agent-status11,674148b9db078460,585665135d311bbd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:44:00.358 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:44:00.359 INFO 12717 --- [host-check-agent-status12,674148b9db078460,b1595972c2045731] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:44:00.360 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:00.361 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:44:00.361 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:44:00.362 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:44:00.363 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:44:00.366 INFO 12717 --- [host-check-agent-status2,674148b9db078460,4b6f515eaa6a1faf] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:44:00.366 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:44:00.371 INFO 12717 --- [host-check-agent-status2,674148b9db078460,4b6f515eaa6a1faf] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:44:00.374 INFO 12717 --- [host-check-agent-status5,674148b9db078460,2dae2df8e8baaa26] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:44:00.374 INFO 12717 --- [host-check-agent-status10,674148b9db078460,819ad72782bb2ae2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:44:00.374 INFO 12717 --- [host-check-agent-status9,674148b9db078460,6641f3dadf66379f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:44:00.375 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:44:00.379 INFO 12717 --- [host-check-agent-status1,674148b9db078460,5fe0906d5884044b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:44:00.380 INFO 12717 --- [host-check-agent-status7,674148b9db078460,a013e607ad4d5d10] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:44:00.381 INFO 12717 --- [host-check-agent-status5,674148b9db078460,2dae2df8e8baaa26] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:44:00.381 INFO 12717 --- [host-check-agent-status10,674148b9db078460,819ad72782bb2ae2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:44:00.381 INFO 12717 --- [host-check-agent-status9,674148b9db078460,6641f3dadf66379f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:44:00.381 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:44:00.381 INFO 12717 --- [host-check-agent-status4,674148b9db078460,36ff88efcc0c6d2a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:44:00.381 INFO 12717 --- [host-check-agent-status11,674148b9db078460,585665135d311bbd] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:44:00.382 INFO 12717 --- [host-check-agent-status6,674148b9db078460,f6d90b29a060aace] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:44:00.383 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:44:00.383 INFO 12717 --- [host-check-agent-status1,674148b9db078460,5fe0906d5884044b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:44:00.384 INFO 12717 --- [host-check-agent-status3,674148b9db078460,9c2e51de8e10c89f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:44:00.384 INFO 12717 --- [host-check-agent-status8,674148b9db078460,476e74fc63118162] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:44:00.385 INFO 12717 --- [host-check-agent-status7,674148b9db078460,a013e607ad4d5d10] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:44:00.385 INFO 12717 --- [host-check-agent-status4,674148b9db078460,36ff88efcc0c6d2a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:44:00.386 INFO 12717 --- [host-check-agent-status6,674148b9db078460,f6d90b29a060aace] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:44:00.386 INFO 12717 --- [host-check-agent-status11,674148b9db078460,585665135d311bbd] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:44:00.386 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:44:00.387 INFO 12717 --- [host-check-agent-status3,674148b9db078460,9c2e51de8e10c89f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:44:00.388 INFO 12717 --- [host-check-agent-status8,674148b9db078460,476e74fc63118162] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:44:00.389 INFO 12717 --- [host-check-agent-status12,674148b9db078460,b1595972c2045731] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:44:00.395 INFO 12717 --- [host-check-agent-status12,674148b9db078460,b1595972c2045731] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:44:00.408 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:00.408 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:44:00.415 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/time, params:null 2025-11-29 08:44:00.473 INFO 12717 --- [host-check-clockdiff1,95eb62dc9b2c413b,b6b1aeeec47c71a7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764377040 0 0 , error= 2025-11-29 08:44:00.474 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=8073, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:44:00.474 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/time, params:null 2025-11-29 08:44:00.477 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/time, params:null 2025-11-29 08:44:00.415 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/time, params:null 2025-11-29 08:44:00.477 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/time, params:null 2025-11-29 08:44:00.478 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/time, params:null 2025-11-29 08:44:00.480 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/time, params:null 2025-11-29 08:44:00.480 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.202.22.52, timestamp=2025-11-29T08:44:00.477187171+08:00 2025-11-29 08:44:00.481 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.202.22.50, timestamp=2025-11-29T08:44:00.479948278+08:00 2025-11-29 08:44:00.481 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.202.22.54, timestamp=2025-11-29T08:44:00.478694890+08:00 2025-11-29 08:44:00.482 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-11-29T08:44:00.480+08:00 2025-11-29 08:44:00.482 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.202.22.51, timestamp=2025-11-29T08:44:00.478853377+08:00 2025-11-29 08:44:00.482 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-11-29T08:44:00.481+08:00 2025-11-29 08:44:00.482 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.202.22.58, timestamp=2025-11-29T08:44:00.481573222+08:00 2025-11-29 08:44:00.483 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2025-11-29T08:44:00.481+08:00 2025-11-29 08:44:00.483 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-11-29T08:44:00.482+08:00 2025-11-29 08:44:00.483 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-11-29T08:44:00.482+08:00 2025-11-29 08:44:00.484 INFO 12717 --- [host-check-agent-status5,674148b9db078460,2dae2df8e8baaa26] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.485 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/time, params:null 2025-11-29 08:44:00.485 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/time, params:null 2025-11-29 08:44:00.486 INFO 12717 --- [host-check-agent-status9,674148b9db078460,6641f3dadf66379f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.486 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-11-29T08:44:00.480+08:00 and status AVAILABLE 2025-11-29 08:44:00.487 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2025-11-29T08:44:00.481+08:00 and status AVAILABLE 2025-11-29 08:44:00.487 INFO 12717 --- [host-check-agent-status10,674148b9db078460,819ad72782bb2ae2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.488 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-11-29T08:44:00.482+08:00 and status AVAILABLE 2025-11-29 08:44:00.488 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/time, params:null 2025-11-29 08:44:00.488 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-11-29T08:44:00.481+08:00 and status AVAILABLE 2025-11-29 08:44:00.488 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=13, ip=10.202.22.49, timestamp=2025-11-29T08:44:00.486583484+08:00 2025-11-29 08:44:00.488 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-11-29T08:44:00.482+08:00 and status AVAILABLE 2025-11-29 08:44:00.488 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.202.22.53, timestamp=2025-11-29T08:44:00.487491688+08:00 2025-11-29 08:44:00.491 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=11, ip=10.202.22.47, timestamp=2025-11-29T08:44:00.490179605+08:00 2025-11-29 08:44:00.492 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-11-29T08:44:00.489+08:00 2025-11-29 08:44:00.492 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=13, lastAvailableTime=2025-11-29T08:44:00.488+08:00 2025-11-29 08:44:00.493 INFO 12717 --- [host-check-agent-status5,674148b9db078460,2dae2df8e8baaa26] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.53 2025-11-29 08:44:00.493 INFO 12717 --- [host-check-agent-status5,674148b9db078460,2dae2df8e8baaa26] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.53 2025-11-29 08:44:00.493 INFO 12717 --- [host-check-agent-status5,674148b9db078460,2dae2df8e8baaa26] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.53 2025-11-29 08:44:00.494 INFO 12717 --- [host-check-agent-status10,674148b9db078460,819ad72782bb2ae2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.47 2025-11-29 08:44:00.494 INFO 12717 --- [host-check-agent-status10,674148b9db078460,819ad72782bb2ae2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.47 2025-11-29 08:44:00.494 INFO 12717 --- [host-check-agent-status10,674148b9db078460,819ad72782bb2ae2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.47 2025-11-29 08:44:00.494 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=11, lastAvailableTime=2025-11-29T08:44:00.491+08:00 2025-11-29 08:44:00.496 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/time, params:null 2025-11-29 08:44:00.497 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-11-29T08:44:00.489+08:00 and status AVAILABLE 2025-11-29 08:44:00.497 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.s.compute.HostAgentDaoManager : Updated agent 12 last available time 2025-11-29T08:44:00.488+08:00 and status AVAILABLE 2025-11-29 08:44:00.498 INFO 12717 --- [host-check-agent-status2,674148b9db078460,4b6f515eaa6a1faf] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.498 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.s.compute.HostAgentDaoManager : Updated agent 11 last available time 2025-11-29T08:44:00.491+08:00 and status AVAILABLE 2025-11-29 08:44:00.498 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/time, params:null 2025-11-29 08:44:00.500 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:00.501 INFO 12717 --- [host-check-agent-status11,674148b9db078460,585665135d311bbd] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.501 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:00.501 INFO 12717 --- [host-check-agent-status1,674148b9db078460,5fe0906d5884044b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.501 INFO 12717 --- [host-check-agent-status4,674148b9db078460,36ff88efcc0c6d2a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.501 INFO 12717 --- [host-check-agent-status7,674148b9db078460,a013e607ad4d5d10] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.501 INFO 12717 --- [host-check-agent-status6,674148b9db078460,f6d90b29a060aace] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.502 INFO 12717 --- [host-check-agent-status2,674148b9db078460,4b6f515eaa6a1faf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.50 2025-11-29 08:44:00.502 INFO 12717 --- [host-check-agent-status2,674148b9db078460,4b6f515eaa6a1faf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.50 2025-11-29 08:44:00.502 INFO 12717 --- [host-check-agent-status2,674148b9db078460,4b6f515eaa6a1faf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.50 2025-11-29 08:44:00.502 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:44:00.505 INFO 12717 --- [host-check-agent-status3,674148b9db078460,9c2e51de8e10c89f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.505 INFO 12717 --- [host-check-agent-status12,674148b9db078460,b1595972c2045731] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.505 INFO 12717 --- [host-check-agent-status4,674148b9db078460,36ff88efcc0c6d2a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.52 2025-11-29 08:44:00.505 INFO 12717 --- [host-check-agent-status4,674148b9db078460,36ff88efcc0c6d2a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.52 2025-11-29 08:44:00.505 INFO 12717 --- [host-check-agent-status4,674148b9db078460,36ff88efcc0c6d2a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.52 2025-11-29 08:44:00.505 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:00.506 INFO 12717 --- [host-check-agent-status6,674148b9db078460,f6d90b29a060aace] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.54 2025-11-29 08:44:00.506 INFO 12717 --- [host-check-agent-status6,674148b9db078460,f6d90b29a060aace] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.54 2025-11-29 08:44:00.506 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:00.506 INFO 12717 --- [host-check-agent-status6,674148b9db078460,f6d90b29a060aace] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.54 2025-11-29 08:44:00.507 INFO 12717 --- [host-check-agent-status8,674148b9db078460,476e74fc63118162] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/agent/status, params:null 2025-11-29 08:44:00.509 INFO 12717 --- [host-check-agent-status3,674148b9db078460,9c2e51de8e10c89f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.51 2025-11-29 08:44:00.509 INFO 12717 --- [host-check-agent-status3,674148b9db078460,9c2e51de8e10c89f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.51 2025-11-29 08:44:00.509 INFO 12717 --- [host-check-agent-status3,674148b9db078460,9c2e51de8e10c89f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.51 2025-11-29 08:44:00.510 INFO 12717 --- [host-check-agent-status12,674148b9db078460,b1595972c2045731] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.49 2025-11-29 08:44:00.510 INFO 12717 --- [host-check-agent-status12,674148b9db078460,b1595972c2045731] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.49 2025-11-29 08:44:00.510 INFO 12717 --- [host-check-agent-status12,674148b9db078460,b1595972c2045731] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.49 2025-11-29 08:44:00.511 INFO 12717 --- [host-check-agent-status1,674148b9db078460,5fe0906d5884044b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.58 2025-11-29 08:44:00.511 INFO 12717 --- [host-check-agent-status1,674148b9db078460,5fe0906d5884044b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.58 2025-11-29 08:44:00.511 INFO 12717 --- [host-check-agent-status1,674148b9db078460,5fe0906d5884044b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.58 2025-11-29 08:44:00.519 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=ocpserver, obClusterId=1743215593 2025-11-29 08:44:00.520 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=ocpserver, obClusterId=1743215593, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"ocpserver","ObClusterId":1743215593,"ObRegion":"ocpserver","ObRegionId":1743215593,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.58:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376673675000}, createTime=2025-03-29T10:43:06+08:00, creator=admin, obVersion=4.2.1.8, packageOperatingSystem=null, loadType=null, tenantCount=3) 2025-11-29 08:44:00.521 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=12, ip=10.202.22.48, timestamp=2025-11-29T08:44:00.517057287+08:00 2025-11-29 08:44:00.523 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=12, lastAvailableTime=2025-11-29T08:44:00.522+08:00 2025-11-29 08:44:00.527 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 13 last available time 2025-11-29T08:44:00.522+08:00 and status AVAILABLE 2025-11-29 08:44:00.553 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=8, ip=10.202.22.59, timestamp=2025-11-29T08:44:00.552250151+08:00 2025-11-29 08:44:00.555 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=8, lastAvailableTime=2025-11-29T08:44:00.553+08:00 2025-11-29 08:44:00.556 INFO 12717 --- [host-check-agent-status7,674148b9db078460,a013e607ad4d5d10] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.59 2025-11-29 08:44:00.556 INFO 12717 --- [host-check-agent-status7,674148b9db078460,a013e607ad4d5d10] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.59 2025-11-29 08:44:00.556 INFO 12717 --- [host-check-agent-status7,674148b9db078460,a013e607ad4d5d10] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.59 2025-11-29 08:44:00.561 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.s.compute.HostAgentDaoManager : Updated agent 8 last available time 2025-11-29T08:44:00.553+08:00 and status AVAILABLE 2025-11-29 08:44:00.561 INFO 12717 --- [Check status for all clusters2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.3.5.1 2025-11-29 08:44:00.616 INFO 12717 --- [host-check-agent-status11,674148b9db078460,585665135d311bbd] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.48 2025-11-29 08:44:00.616 INFO 12717 --- [host-check-agent-status11,674148b9db078460,585665135d311bbd] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.48 2025-11-29 08:44:00.616 INFO 12717 --- [host-check-agent-status11,674148b9db078460,585665135d311bbd] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.48 2025-11-29 08:44:00.633 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=9, ip=10.202.22.60, timestamp=2025-11-29T08:44:00.626327893+08:00 2025-11-29 08:44:00.635 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=10, ip=10.202.22.61, timestamp=2025-11-29T08:44:00.630997854+08:00 2025-11-29 08:44:00.636 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=9, lastAvailableTime=2025-11-29T08:44:00.633+08:00 2025-11-29 08:44:00.636 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=10, lastAvailableTime=2025-11-29T08:44:00.635+08:00 2025-11-29 08:44:00.638 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-11-29 08:44:00.639 INFO 12717 --- [host-check-agent-status8,674148b9db078460,476e74fc63118162] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.60 2025-11-29 08:44:00.639 INFO 12717 --- [host-check-agent-status9,674148b9db078460,6641f3dadf66379f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.61 2025-11-29 08:44:00.639 INFO 12717 --- [host-check-agent-status8,674148b9db078460,476e74fc63118162] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.60 2025-11-29 08:44:00.639 INFO 12717 --- [host-check-agent-status9,674148b9db078460,6641f3dadf66379f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.61 2025-11-29 08:44:00.639 INFO 12717 --- [host-check-agent-status9,674148b9db078460,6641f3dadf66379f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.61 2025-11-29 08:44:00.639 INFO 12717 --- [host-check-agent-status8,674148b9db078460,476e74fc63118162] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.60 2025-11-29 08:44:00.641 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2025-11-29T08:44:00.635+08:00 and status AVAILABLE 2025-11-29 08:44:00.643 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.s.compute.HostAgentDaoManager : Updated agent 10 last available time 2025-11-29T08:44:00.633+08:00 and status AVAILABLE 2025-11-29 08:44:00.645 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-11-29 08:44:00.645 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=8 2025-11-29 08:44:00.645 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2025-11-29 08:44:00.645 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=12 2025-11-29 08:44:00.645 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=11 2025-11-29 08:44:00.645 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-11-29 08:44:00.646 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=13 2025-11-29 08:44:00.646 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:00.646 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:00.647 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-11-29 08:44:00.649 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-11-29 08:44:00.656 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=10 2025-11-29 08:44:00.683 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=9 2025-11-29 08:44:00.724 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:44:00.725 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:44:00.726 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:44:00.726 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:44:00.726 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:44:00.726 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:44:00.727 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:44:00.727 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:44:00.727 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:44:00.727 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:44:00.728 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:44:00.728 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:44:00.728 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:44:00.729 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:44:00.729 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:44:00.729 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:44:00.730 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:44:00.731 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:44:00.733 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:44:00.734 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:44:00.737 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:44:00.738 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:44:00.751 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:00.751 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:00.752 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:44:00.754 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:44:00.761 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.762 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.762 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.764 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.764 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.765 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.765 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.766 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.772 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.773 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.792 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.807 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:00.807 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:30+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:00.812 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:00.816 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-11-29 08:44:00.820 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=16 2025-11-29 08:44:00.830 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-11-29 08:44:00.838 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-11-29 08:44:00.840 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=34 2025-11-29 08:44:00.843 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-11-29 08:44:00.845 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=31 2025-11-29 08:44:00.852 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=16 2025-11-29 08:44:00.852 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-11-29 08:44:00.859 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-11-29 08:44:00.860 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=34 2025-11-29 08:44:00.861 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=31 2025-11-29 08:44:00.862 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-11-29 08:44:00.866 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-11-29 08:44:00.869 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-11-29 08:44:00.886 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-11-29 08:44:00.959 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=22 2025-11-29 08:44:00.977 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=22 2025-11-29 08:44:01.005 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:01.005 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:01.035 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.036 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.037 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.037 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.037 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.037 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.038 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.038 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.039 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.042 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:01.043 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:44:01.067 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=25 2025-11-29 08:44:01.071 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:01.071 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:01.086 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=25 2025-11-29 08:44:01.086 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=17 2025-11-29 08:44:01.086 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=28 2025-11-29 08:44:01.086 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-11-29 08:44:01.097 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=35 2025-11-29 08:44:01.099 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:01.099 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:42+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:01.100 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=28 2025-11-29 08:44:01.101 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=17 2025-11-29 08:44:01.102 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-11-29 08:44:01.102 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-11-29 08:44:01.104 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=32 2025-11-29 08:44:01.107 INFO 12717 --- [ocp-schedules-17,d5fb4cfb55417836,5d7bdfce2a5fb836] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration auto replace status 2025-11-29 08:44:01.112 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=35 2025-11-29 08:44:01.114 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-11-29 08:44:01.116 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-11-29 08:44:01.116 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-11-29 08:44:01.120 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=32 2025-11-29 08:44:01.120 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-11-29 08:44:01.122 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-11-29 08:44:01.124 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.129 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.131 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-11-29 08:44:01.132 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-11-29 08:44:01.134 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.140 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.141 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:01.141 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:01.144 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.152 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.154 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.158 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.160 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.161 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.180 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:01.183 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=18 2025-11-29 08:44:01.195 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-11-29 08:44:01.199 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-11-29 08:44:01.199 INFO 12717 --- [host-check-host-status2,b0f2e771b9f8ac93,214901e6729a7621] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=18 2025-11-29 08:44:01.205 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:01.206 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:01.207 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:44:01.208 INFO 12717 --- [host-check-host-status4,b0f2e771b9f8ac93,c0eaeebcb040842e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-11-29 08:44:01.209 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=36 2025-11-29 08:44:01.213 INFO 12717 --- [host-check-host-status6,b0f2e771b9f8ac93,97ad12a9028f0621] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-11-29 08:44:01.214 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-11-29 08:44:01.214 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-11-29 08:44:01.219 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=33 2025-11-29 08:44:01.226 INFO 12717 --- [host-check-host-status12,b0f2e771b9f8ac93,690bdea02d9f2d6a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=36 2025-11-29 08:44:01.227 INFO 12717 --- [host-check-host-status3,b0f2e771b9f8ac93,4956754b06d8f517] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-11-29 08:44:01.227 INFO 12717 --- [host-check-host-status5,b0f2e771b9f8ac93,0834c04128d14d23] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-11-29 08:44:01.233 INFO 12717 --- [host-check-host-status10,b0f2e771b9f8ac93,d8796a542be8b3b6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=33 2025-11-29 08:44:01.238 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-11-29 08:44:01.249 INFO 12717 --- [host-check-host-status1,b0f2e771b9f8ac93,6c3b0654c2f389a7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-11-29 08:44:01.297 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=23 2025-11-29 08:44:01.312 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=23 2025-11-29 08:44:01.368 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.380 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=26 2025-11-29 08:44:01.394 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=26 2025-11-29 08:44:01.405 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=29 2025-11-29 08:44:01.420 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=29 2025-11-29 08:44:01.422 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.444 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:01.451 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster DB 2025-11-29 08:44:01.457 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 2025-11-29 08:44:01.463 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:44:01.466 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:44:01.480 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=27 2025-11-29 08:44:01.493 INFO 12717 --- [host-check-host-status9,b0f2e771b9f8ac93,aa57504d66b80064] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=27 2025-11-29 08:44:01.497 INFO 12717 --- [Sync all clusters2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:44:01.547 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:01.548 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:01.551 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=37 2025-11-29 08:44:01.560 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=24 2025-11-29 08:44:01.570 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=37 2025-11-29 08:44:01.580 INFO 12717 --- [host-check-host-status7,b0f2e771b9f8ac93,fba78971f9747bff] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=24 2025-11-29 08:44:01.688 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 got ObProcessInfoResult(pid=2447, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:44:01.698 INFO 12717 --- [ocp-async-28,38c484d090a99fd3,1ddbbfd2678d5e2f] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:01.707 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=30 2025-11-29 08:44:01.714 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:01.719 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:01.720 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:01.721 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:44:01.723 INFO 12717 --- [host-check-host-status8,b0f2e771b9f8ac93,ebe769c814249cab] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=30 2025-11-29 08:44:01.748 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=DB, obClusterId=1743215594 2025-11-29 08:44:01.748 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=DB, obClusterId=1743215594, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=CREATED, syncStatus=VALID, rootserverJson={"ObCluster":"DB","ObClusterId":1743215594,"ObRegion":"DB","ObRegionId":1743215594,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.52:2882","role":"LEADER","sql_port":2881},{"address":"10.202.22.49:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.202.22.61:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376629943000}, createTime=2025-03-29T11:15:04+08:00, creator=admin, obVersion=4.3.5.1, packageOperatingSystem=el8, loadType=HTAP, tenantCount=7) 2025-11-29 08:44:01.798 WARN 12717 --- [host-check-clockdiff2,95eb62dc9b2c413b,ceb462550d838613] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.50], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:01.798 ERROR 12717 --- [host-check-clockdiff2,95eb62dc9b2c413b,ceb462550d838613] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.50] 2025-11-29 08:44:01.798 WARN 12717 --- [host-check-clockdiff2,95eb62dc9b2c413b,ceb462550d838613] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.202.22.50, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.50 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:01.806 WARN 12717 --- [host-check-clockdiff10,95eb62dc9b2c413b,f892e6e783615d13] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.47], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:01.806 ERROR 12717 --- [host-check-clockdiff10,95eb62dc9b2c413b,f892e6e783615d13] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.47] 2025-11-29 08:44:01.806 WARN 12717 --- [host-check-clockdiff10,95eb62dc9b2c413b,f892e6e783615d13] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=11, ip=10.202.22.47, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.47 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:01.984 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=38 2025-11-29 08:44:01.996 WARN 12717 --- [host-check-clockdiff3,95eb62dc9b2c413b,147b12f73ac74c74] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.51], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:01.996 ERROR 12717 --- [host-check-clockdiff3,95eb62dc9b2c413b,147b12f73ac74c74] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.51] 2025-11-29 08:44:01.997 WARN 12717 --- [host-check-clockdiff3,95eb62dc9b2c413b,147b12f73ac74c74] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.202.22.51, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.51 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:01.999 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=38 2025-11-29 08:44:02.036 WARN 12717 --- [host-check-clockdiff4,95eb62dc9b2c413b,ce11a78b3d0285dd] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.52], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:02.036 ERROR 12717 --- [host-check-clockdiff4,95eb62dc9b2c413b,ce11a78b3d0285dd] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.52] 2025-11-29 08:44:02.036 WARN 12717 --- [host-check-clockdiff4,95eb62dc9b2c413b,ce11a78b3d0285dd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.202.22.52, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.52 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:02.159 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:02.182 INFO 12717 --- [host-check-clockdiff12,95eb62dc9b2c413b,f77bc897dda4fef1] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764377042 -2 -1 , error= 2025-11-29 08:44:02.192 WARN 12717 --- [host-check-clockdiff6,95eb62dc9b2c413b,87a7c3bf9e62fc2c] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.54], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:02.193 ERROR 12717 --- [host-check-clockdiff6,95eb62dc9b2c413b,87a7c3bf9e62fc2c] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.54] 2025-11-29 08:44:02.193 WARN 12717 --- [host-check-clockdiff6,95eb62dc9b2c413b,87a7c3bf9e62fc2c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.202.22.54, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.54 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:02.362 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=39 2025-11-29 08:44:02.379 INFO 12717 --- [host-check-host-status11,b0f2e771b9f8ac93,23190da245f5855c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=39 2025-11-29 08:44:02.407 INFO 12717 --- [prometheus-minute-schedule-1,6972e1807af2a006,93cb5462e7d228f6] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:02.421 INFO 12717 --- [prometheus-minute-schedule-1,6972e1807af2a006,93cb5462e7d228f6] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:02.825 INFO 12717 --- [pool-overview-container-1,b6b8e753cfae423f,a161ff722a277073] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-11-29 08:44:02.825 INFO 12717 --- [pool-overview-container-3,284fa21bb779fbf2,73924d380e314651] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-11-29 08:44:02.847 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,fa2ba8bfd9a299eb,9008a3e0dfc8006c] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=fa2ba8bfd9a299eb, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:02.848 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-83,fa2ba8bfd9a299eb,9008a3e0dfc8006c] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=fa2ba8bfd9a299eb, duration=2 ms] 2025-11-29 08:44:02.857 INFO 12717 --- [pool-11-thread-15,b6b8e753cfae423f,6b2b0deedc382b36] c.o.o.p.t.i.ObTransactionDiagServiceImpl : begin to get cluster trans stat 2025-11-29 08:44:02.858 INFO 12717 --- [pool-11-thread-13,b6b8e753cfae423f,718ef63e96096da8] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get Cluster Sql Stat 2025-11-29 08:44:02.858 INFO 12717 --- [pool-11-thread-14,b6b8e753cfae423f,63ca90833ba460d8] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get cluster session stat 2025-11-29 08:44:02.862 INFO 12717 --- [pool-11-thread-16,b6b8e753cfae423f,2dd6e0c29772d0ba] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get outline history 2025-11-29 08:44:02.866 INFO 12717 --- [ForkJoinPool-7-worker-9,,] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-11-29 08:44:02.866 INFO 12717 --- [ForkJoinPool-7-worker-0,,] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-11-29 08:44:02.867 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get tenant session stat of DB 2025-11-29 08:44:02.867 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.p.o.i.OasSessionStatServiceImpl : begin to get session of DB 2025-11-29 08:44:02.868 INFO 12717 --- [ForkJoinPool-7-worker-9,b6b8e753cfae423f,3d2762804274e547] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get tenant session stat of ocpserver 2025-11-29 08:44:02.868 INFO 12717 --- [ForkJoinPool-7-worker-9,b6b8e753cfae423f,3d2762804274e547] c.o.o.p.o.i.OasSessionStatServiceImpl : begin to get session of ocpserver 2025-11-29 08:44:02.869 INFO 12717 --- [pool-overview-container-7,36e94b36019d7bb5,c729a0d38994d1a4] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:02.871 INFO 12717 --- [pool-overview-container-7,36e94b36019d7bb5,c729a0d38994d1a4] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT SVR_IP, SVR_PORT, ZONE, CPU_CAPACITY, CPU_CAPACITY_MAX, CPU_ASSIGNED, CPU_ASSIGNED_MAX, MEM_CAPACITY, MEM_CAPACITY AS MEM_CAPACITY_MAX, MEM_ASSIGNED, MEM_ASSIGNED AS MEM_ASSIGNED_MAX, LOG_DISK_CAPACITY, LOG_DISK_ASSIGNED, LOG_DISK_IN_USE, DATA_DISK_CAPACITY, DATA_DISK_IN_USE FROM oceanbase.GV$OB_SERVERS 2025-11-29 08:44:02.892 INFO 12717 --- [pool-11-thread-15,b6b8e753cfae423f,6b2b0deedc382b36] c.o.o.p.t.i.ObTransactionDiagServiceImpl : begin to attach trans id 2025-11-29 08:44:02.892 INFO 12717 --- [pool-11-thread-15,b6b8e753cfae423f,6b2b0deedc382b36] c.o.o.p.t.i.ObTransactionDiagServiceImpl : finish fetching trans data 2025-11-29 08:44:02.904 INFO 12717 --- [pool-11-thread-13,b6b8e753cfae423f,718ef63e96096da8] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get cluster slow Sql Stat 2025-11-29 08:44:02.947 WARN 12717 --- [ForkJoinPool-7-worker-9,b6b8e753cfae423f,3d2762804274e547] c.o.o.p.oas.internal.OasStatServiceImpl : Failed to get session info of :ocpserver com.oceanbase.ocp.service.profile.exception.CredentialMissingException: [OCP CredentialMissingException]: status=428 PRECONDITION_REQUIRED, errorCode=IAM_CREDENTIAL_REQUIRED, args=ocpserver,ocp_meta,root at com.oceanbase.ocp.service.profile.exception.CredentialMissingException.ob(CredentialMissingException.java:34) at com.oceanbase.ocp.service.credential.CredentialServiceImpl.acquireJdbcPasswordInner(CredentialServiceImpl.java:166) at com.oceanbase.ocp.service.credential.CredentialServiceImpl.acquireObPassword(CredentialServiceImpl.java:143) at com.oceanbase.ocp.service.credential.CredentialServiceImpl$$FastClassBySpringCGLIB$$6855c59e.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386) at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703) at com.oceanbase.ocp.service.credential.CredentialServiceImpl$$EnhancerBySpringCGLIB$$4649d365.acquireObPassword() at com.oceanbase.ocp.service.ob.obsdk.factory.ConnectPropertiesBuilder.acquirePassword(ConnectPropertiesBuilder.java:204) at com.oceanbase.ocp.service.ob.obsdk.factory.ConnectPropertiesBuilder.buildConnectProperties(ConnectPropertiesBuilder.java:97) at com.oceanbase.ocp.service.ob.obsdk.factory.ObAccessorFactory.buildConnectProperties(ObAccessorFactory.java:180) at com.oceanbase.ocp.service.ob.obsdk.factory.ObAccessorFactory.createObAccessor(ObAccessorFactory.java:57) at com.oceanbase.ocp.obops.internal.tenant.TenantSessionServiceImpl.lambda$countSession$1(TenantSessionServiceImpl.java:243) at java.lang.Iterable.forEach(Iterable.java:75) at com.oceanbase.ocp.obops.internal.tenant.TenantSessionServiceImpl.countSession(TenantSessionServiceImpl.java:215) at com.oceanbase.ocp.perf.oas.internal.OasSessionStatServiceImpl.getTenantSessionStat(OasSessionStatServiceImpl.java:71) at com.oceanbase.ocp.perf.oas.internal.OasStatServiceImpl.lambda$null$8(OasStatServiceImpl.java:281) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290) at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) 2025-11-29 08:44:02.955 INFO 12717 --- [ForkJoinPool-7-worker-11,,] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-11-29 08:44:02.955 INFO 12717 --- [ForkJoinPool-7-worker-2,,] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-11-29 08:44:02.982 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:02.983 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state = 'ACTIVE' 2025-11-29 08:44:02.998 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.001 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2025-11-29 08:44:03.003 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.004 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2025-11-29 08:44:03.033 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=lhp6796, dbUser=root 2025-11-29 08:44:03.033 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.035 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2025-11-29 08:44:03.036 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.038 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2025-11-29 08:44:03.047 INFO 12717 --- [refresh-tenant-resource-cache-5,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.053 INFO 12717 --- [refresh-tenant-resource-cache-5,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(60000000) */ time_to_usec(t1.MODIFY_TIME) AS UPDATE_TIME, t1.RESOURCE_POOL_ID, t1.NAME, t1.UNIT_COUNT, t1.UNIT_CONFIG_ID, t1.ZONE_LIST, t1.TENANT_ID, t1.REPLICA_TYPE, t2.NAME AS UNIT_CONFIG_NAME, t2.MAX_CPU, t2.MIN_CPU, t2.MEMORY_SIZE AS MAX_MEMORY, t2.MEMORY_SIZE AS MIN_MEMORY, t2.LOG_DISK_SIZE, t2.MAX_IOPS, t2.MIN_IOPS, t2.IOPS_WEIGHT FROM oceanbase.DBA_OB_RESOURCE_POOLS AS t1 JOIN oceanbase.DBA_OB_UNIT_CONFIGS AS t2 ON t1.UNIT_CONFIG_ID = t2.UNIT_CONFIG_ID AND t1.tenant_id = ?, args: [1010] 2025-11-29 08:44:03.079 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.081 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2025-11-29 08:44:03.103 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=nextcloud, dbUser=root 2025-11-29 08:44:03.104 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.105 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2025-11-29 08:44:03.106 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.107 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2025-11-29 08:44:03.121 INFO 12717 --- [refresh-tenant-resource-cache-5,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:03.131 INFO 12717 --- [refresh-tenant-resource-cache-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.132 INFO 12717 --- [ocp-async-1,4b874633c76827bf,d63543566e987f1d] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:44:03.132 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.133 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state = 'ACTIVE' 2025-11-29 08:44:03.133 INFO 12717 --- [refresh-tenant-resource-cache-5,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.136 INFO 12717 --- [refresh-tenant-resource-cache-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(60000000) */ time_to_usec(t1.MODIFY_TIME) AS UPDATE_TIME, t1.RESOURCE_POOL_ID, t1.NAME, t1.UNIT_COUNT, t1.UNIT_CONFIG_ID, t1.ZONE_LIST, t1.TENANT_ID, t1.REPLICA_TYPE, t2.NAME AS UNIT_CONFIG_NAME, t2.MAX_CPU, t2.MIN_CPU, t2.MEMORY_SIZE AS MAX_MEMORY, t2.MEMORY_SIZE AS MIN_MEMORY, t2.LOG_DISK_SIZE, t2.MAX_IOPS, t2.MIN_IOPS, t2.IOPS_WEIGHT FROM oceanbase.DBA_OB_RESOURCE_POOLS AS t1 JOIN oceanbase.DBA_OB_UNIT_CONFIGS AS t2 ON t1.UNIT_CONFIG_ID = t2.UNIT_CONFIG_ID AND t1.tenant_id = ?, args: [1018] 2025-11-29 08:44:03.139 INFO 12717 --- [refresh-tenant-resource-cache-5,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(5000000) */ t1.UNIT_ID, t1.RESOURCE_POOL_ID, t1.UNIT_CONFIG_ID, t1.ZONE, t1.SVR_IP, t1.SVR_PORT, t1.STATUS, t2.REPLICA_TYPE, t2.TENANT_ID, t3.TENANT_NAME, t2.NAME AS RESOURCE_POOL_NAME, t2.MODIFY_TIME AS RESOURCE_POOL_UPDATE_TIME, t1.MIGRATE_FROM_SVR_IP, t1.MIGRATE_FROM_SVR_PORT, t1.MANUAL_MIGRATE FROM oceanbase.DBA_OB_UNITS AS t1 JOIN oceanbase.DBA_OB_RESOURCE_POOLS AS t2 ON t1.RESOURCE_POOL_ID = t2.RESOURCE_POOL_ID JOIN oceanbase.DBA_OB_TENANTS t3 ON t2.TENANT_ID = t3.TENANT_ID WHERE 1=1 AND t2.tenant_id = ?, args: [1010] 2025-11-29 08:44:03.141 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.143 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2025-11-29 08:44:03.168 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=oms, dbUser=root 2025-11-29 08:44:03.169 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.170 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2025-11-29 08:44:03.171 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.172 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2025-11-29 08:44:03.196 INFO 12717 --- [refresh-tenant-resource-cache-10,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.201 INFO 12717 --- [refresh-tenant-resource-cache-10,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(60000000) */ time_to_usec(t1.MODIFY_TIME) AS UPDATE_TIME, t1.RESOURCE_POOL_ID, t1.NAME, t1.UNIT_COUNT, t1.UNIT_CONFIG_ID, t1.ZONE_LIST, t1.TENANT_ID, t1.REPLICA_TYPE, t2.NAME AS UNIT_CONFIG_NAME, t2.MAX_CPU, t2.MIN_CPU, t2.MEMORY_SIZE AS MAX_MEMORY, t2.MEMORY_SIZE AS MIN_MEMORY, t2.LOG_DISK_SIZE, t2.MAX_IOPS, t2.MIN_IOPS, t2.IOPS_WEIGHT FROM oceanbase.DBA_OB_RESOURCE_POOLS AS t1 JOIN oceanbase.DBA_OB_UNIT_CONFIGS AS t2 ON t1.UNIT_CONFIG_ID = t2.UNIT_CONFIG_ID AND t1.tenant_id = ?, args: [1014] 2025-11-29 08:44:03.205 INFO 12717 --- [refresh-tenant-resource-cache-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:03.205 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.206 INFO 12717 --- [refresh-tenant-resource-cache-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.208 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2025-11-29 08:44:03.211 INFO 12717 --- [refresh-tenant-resource-cache-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(5000000) */ t1.UNIT_ID, t1.RESOURCE_POOL_ID, t1.UNIT_CONFIG_ID, t1.ZONE, t1.SVR_IP, t1.SVR_PORT, t1.STATUS, t2.REPLICA_TYPE, t2.TENANT_ID, t3.TENANT_NAME, t2.NAME AS RESOURCE_POOL_NAME, t2.MODIFY_TIME AS RESOURCE_POOL_UPDATE_TIME, t1.MIGRATE_FROM_SVR_IP, t1.MIGRATE_FROM_SVR_PORT, t1.MANUAL_MIGRATE FROM oceanbase.DBA_OB_UNITS AS t1 JOIN oceanbase.DBA_OB_RESOURCE_POOLS AS t2 ON t1.RESOURCE_POOL_ID = t2.RESOURCE_POOL_ID JOIN oceanbase.DBA_OB_TENANTS t3 ON t2.TENANT_ID = t3.TENANT_ID WHERE 1=1 AND t2.tenant_id = ?, args: [1018] 2025-11-29 08:44:03.232 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=pipeline, dbUser=root 2025-11-29 08:44:03.234 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.234 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.235 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2025-11-29 08:44:03.237 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2025-11-29 08:44:03.239 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.240 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2025-11-29 08:44:03.269 INFO 12717 --- [refresh-tenant-resource-cache-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.270 INFO 12717 --- [pool-11-thread-13,b6b8e753cfae423f,718ef63e96096da8] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get cluster anomaly sql Stat 2025-11-29 08:44:03.274 INFO 12717 --- [refresh-tenant-resource-cache-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(60000000) */ time_to_usec(t1.MODIFY_TIME) AS UPDATE_TIME, t1.RESOURCE_POOL_ID, t1.NAME, t1.UNIT_COUNT, t1.UNIT_CONFIG_ID, t1.ZONE_LIST, t1.TENANT_ID, t1.REPLICA_TYPE, t2.NAME AS UNIT_CONFIG_NAME, t2.MAX_CPU, t2.MIN_CPU, t2.MEMORY_SIZE AS MAX_MEMORY, t2.MEMORY_SIZE AS MIN_MEMORY, t2.LOG_DISK_SIZE, t2.MAX_IOPS, t2.MIN_IOPS, t2.IOPS_WEIGHT FROM oceanbase.DBA_OB_RESOURCE_POOLS AS t1 JOIN oceanbase.DBA_OB_UNIT_CONFIGS AS t2 ON t1.UNIT_CONFIG_ID = t2.UNIT_CONFIG_ID AND t1.tenant_id = ?, args: [1020] 2025-11-29 08:44:03.284 INFO 12717 --- [refresh-tenant-resource-cache-10,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:03.285 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:03.285 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:03.285 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:03.285 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:03.285 INFO 12717 --- [refresh-tenant-resource-cache-10,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.286 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:03.286 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:03.286 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:03.286 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:03.286 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.288 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2025-11-29 08:44:03.294 INFO 12717 --- [refresh-tenant-resource-cache-10,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(5000000) */ t1.UNIT_ID, t1.RESOURCE_POOL_ID, t1.UNIT_CONFIG_ID, t1.ZONE, t1.SVR_IP, t1.SVR_PORT, t1.STATUS, t2.REPLICA_TYPE, t2.TENANT_ID, t3.TENANT_NAME, t2.NAME AS RESOURCE_POOL_NAME, t2.MODIFY_TIME AS RESOURCE_POOL_UPDATE_TIME, t1.MIGRATE_FROM_SVR_IP, t1.MIGRATE_FROM_SVR_PORT, t1.MANUAL_MIGRATE FROM oceanbase.DBA_OB_UNITS AS t1 JOIN oceanbase.DBA_OB_RESOURCE_POOLS AS t2 ON t1.RESOURCE_POOL_ID = t2.RESOURCE_POOL_ID JOIN oceanbase.DBA_OB_TENANTS t3 ON t2.TENANT_ID = t3.TENANT_ID WHERE 1=1 AND t2.tenant_id = ?, args: [1014] 2025-11-29 08:44:03.296 INFO 12717 --- [pool-11-thread-13,b6b8e753cfae423f,718ef63e96096da8] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching cluster sql stat 2025-11-29 08:44:03.314 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=root 2025-11-29 08:44:03.314 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:03.314 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:03.314 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:03.314 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:03.317 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:03.317 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:03.317 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:03.317 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:03.321 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.326 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2025-11-29 08:44:03.329 WARN 12717 --- [host-check-clockdiff5,95eb62dc9b2c413b,7997d287f671b036] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.53], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:03.329 ERROR 12717 --- [host-check-clockdiff5,95eb62dc9b2c413b,7997d287f671b036] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.53] 2025-11-29 08:44:03.329 WARN 12717 --- [host-check-clockdiff5,95eb62dc9b2c413b,7997d287f671b036] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.202.22.53, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.53 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:03.338 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:03.338 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:03.338 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:03.338 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:03.338 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.340 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2025-11-29 08:44:03.348 WARN 12717 --- [host-check-clockdiff7,95eb62dc9b2c413b,a38482e3e6b8541e] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.59], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:03.348 ERROR 12717 --- [host-check-clockdiff7,95eb62dc9b2c413b,a38482e3e6b8541e] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.59] 2025-11-29 08:44:03.348 WARN 12717 --- [host-check-clockdiff7,95eb62dc9b2c413b,a38482e3e6b8541e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=8, ip=10.202.22.59, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.59 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:03.352 INFO 12717 --- [refresh-tenant-resource-cache-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:03.353 INFO 12717 --- [refresh-tenant-resource-cache-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.359 INFO 12717 --- [refresh-tenant-resource-cache-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(5000000) */ t1.UNIT_ID, t1.RESOURCE_POOL_ID, t1.UNIT_CONFIG_ID, t1.ZONE, t1.SVR_IP, t1.SVR_PORT, t1.STATUS, t2.REPLICA_TYPE, t2.TENANT_ID, t3.TENANT_NAME, t2.NAME AS RESOURCE_POOL_NAME, t2.MODIFY_TIME AS RESOURCE_POOL_UPDATE_TIME, t1.MIGRATE_FROM_SVR_IP, t1.MIGRATE_FROM_SVR_PORT, t1.MANUAL_MIGRATE FROM oceanbase.DBA_OB_UNITS AS t1 JOIN oceanbase.DBA_OB_RESOURCE_POOLS AS t2 ON t1.RESOURCE_POOL_ID = t2.RESOURCE_POOL_ID JOIN oceanbase.DBA_OB_TENANTS t3 ON t2.TENANT_ID = t3.TENANT_ID WHERE 1=1 AND t2.tenant_id = ?, args: [1020] 2025-11-29 08:44:03.363 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.363 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state = 'ACTIVE' 2025-11-29 08:44:03.378 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.380 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2025-11-29 08:44:03.401 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=wecom_bot, dbUser=root 2025-11-29 08:44:03.402 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.404 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2025-11-29 08:44:03.405 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.406 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2025-11-29 08:44:03.409 INFO 12717 --- [ocp-schedules-4,c32d3e65fcf5543a,4d80b423dbbcae79] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:44:03.421 INFO 12717 --- [alarm-detect-12,c32d3e65fcf5543a,65b177d832785cc6] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008342, data is empty. 2025-11-29 08:44:03.421 INFO 12717 --- [alarm-detect-1,c32d3e65fcf5543a,48d071202e38ca56] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008342, data is empty. 2025-11-29 08:44:03.423 INFO 12717 --- [alarm-detect-4,c32d3e65fcf5543a,87e9c6ad4fa2ad39] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008342, data is empty. 2025-11-29 08:44:03.429 INFO 12717 --- [alarm-detect-12,c32d3e65fcf5543a,65b177d832785cc6] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008342, data is empty. 2025-11-29 08:44:03.429 INFO 12717 --- [alarm-detect-1,c32d3e65fcf5543a,48d071202e38ca56] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008342, data is empty. 2025-11-29 08:44:03.430 INFO 12717 --- [alarm-detect-4,c32d3e65fcf5543a,87e9c6ad4fa2ad39] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008342, data is empty. 2025-11-29 08:44:03.438 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.440 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2025-11-29 08:44:03.442 WARN 12717 --- [scanner-11213815135-1,c32d3e65fcf5543a,4b2f717ecc4815df] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:44:03.449 INFO 12717 --- [refresh-tenant-resource-cache-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.457 INFO 12717 --- [refresh-tenant-resource-cache-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(60000000) */ time_to_usec(t1.MODIFY_TIME) AS UPDATE_TIME, t1.RESOURCE_POOL_ID, t1.NAME, t1.UNIT_COUNT, t1.UNIT_CONFIG_ID, t1.ZONE_LIST, t1.TENANT_ID, t1.REPLICA_TYPE, t2.NAME AS UNIT_CONFIG_NAME, t2.MAX_CPU, t2.MIN_CPU, t2.MEMORY_SIZE AS MAX_MEMORY, t2.MEMORY_SIZE AS MIN_MEMORY, t2.LOG_DISK_SIZE, t2.MAX_IOPS, t2.MIN_IOPS, t2.IOPS_WEIGHT FROM oceanbase.DBA_OB_RESOURCE_POOLS AS t1 JOIN oceanbase.DBA_OB_UNIT_CONFIGS AS t2 ON t1.UNIT_CONFIG_ID = t2.UNIT_CONFIG_ID AND t1.tenant_id = ?, args: [1024] 2025-11-29 08:44:03.462 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.464 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2025-11-29 08:44:03.492 INFO 12717 --- [alarm-detect-2,c32d3e65fcf5543a,3331b2375b787418] c.o.o.a.c.detect.AlarmRuleDetectState : skipPending, alarmType=ocp_remote_server_time_diff_too_large, target=alarm_template_id=3:host=10.202.22.59-1, activeAt=null 2025-11-29 08:44:03.492 INFO 12717 --- [alarm-detect-2,c32d3e65fcf5543a,3331b2375b787418] c.o.o.a.c.detect.AlarmRuleDetectState : setResolve, alarmType=ocp_remote_server_time_diff_too_large, target=alarm_template_id=3:host=10.202.22.59-1, validUntil=2025-11-29T08:05:31.435+08:00 2025-11-29 08:44:03.492 INFO 12717 --- [alarm-detect-2,c32d3e65fcf5543a,3331b2375b787418] c.o.o.a.core.detect.AlarmRuleDetector : detectByMetric done, alarmType=ocp_remote_server_time_diff_too_large, needSendingCount=1, detectAlarmsCount=1, matchedTargets=[alarm_template_id=3:host=10.202.22.59-1] 2025-11-29 08:44:03.492 INFO 12717 --- [alarm-detect-2,c32d3e65fcf5543a,3331b2375b787418] c.o.o.a.core.task.AlarmDetectMetricTask : alarm ocp_remote_server_time_diff_too_large, target alarm_template_id=3:host=10.202.22.59 is not Active 2025-11-29 08:44:03.496 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=zto, dbUser=root 2025-11-29 08:44:03.497 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.499 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2025-11-29 08:44:03.501 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.502 INFO 12717 --- [ForkJoinPool-7-worker-2,b6b8e753cfae423f,6769b0b6fa32c4b5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2025-11-29 08:44:03.518 INFO 12717 --- [pool-11-thread-16,b6b8e753cfae423f,2dd6e0c29772d0ba] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching outline history 2025-11-29 08:44:03.545 INFO 12717 --- [alarm-detect-7,c32d3e65fcf5543a,6069077c08b73c8d] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008342, data is empty. 2025-11-29 08:44:03.551 INFO 12717 --- [refresh-tenant-resource-cache-6,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.555 INFO 12717 --- [alarm-detect-7,c32d3e65fcf5543a,6069077c08b73c8d] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008342, data is empty. 2025-11-29 08:44:03.555 INFO 12717 --- [alarm-detect-2,c32d3e65fcf5543a,3331b2375b787418] c.o.o.alarm.service.OcpAlarmServiceImpl : alarm can not be resoled because alarm target is not found, alarmType=ocp_remote_server_time_diff_too_large, target=alarm_template_id=3:host=10.202.22.59 2025-11-29 08:44:03.559 INFO 12717 --- [refresh-tenant-resource-cache-6,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(60000000) */ time_to_usec(t1.MODIFY_TIME) AS UPDATE_TIME, t1.RESOURCE_POOL_ID, t1.NAME, t1.UNIT_COUNT, t1.UNIT_CONFIG_ID, t1.ZONE_LIST, t1.TENANT_ID, t1.REPLICA_TYPE, t2.NAME AS UNIT_CONFIG_NAME, t2.MAX_CPU, t2.MIN_CPU, t2.MEMORY_SIZE AS MAX_MEMORY, t2.MEMORY_SIZE AS MIN_MEMORY, t2.LOG_DISK_SIZE, t2.MAX_IOPS, t2.MIN_IOPS, t2.IOPS_WEIGHT FROM oceanbase.DBA_OB_RESOURCE_POOLS AS t1 JOIN oceanbase.DBA_OB_UNIT_CONFIGS AS t2 ON t1.UNIT_CONFIG_ID = t2.UNIT_CONFIG_ID AND t1.tenant_id = ?, args: [1022] 2025-11-29 08:44:03.582 INFO 12717 --- [refresh-tenant-resource-cache-3,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:03.585 INFO 12717 --- [refresh-tenant-resource-cache-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.590 INFO 12717 --- [refresh-tenant-resource-cache-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(5000000) */ t1.UNIT_ID, t1.RESOURCE_POOL_ID, t1.UNIT_CONFIG_ID, t1.ZONE, t1.SVR_IP, t1.SVR_PORT, t1.STATUS, t2.REPLICA_TYPE, t2.TENANT_ID, t3.TENANT_NAME, t2.NAME AS RESOURCE_POOL_NAME, t2.MODIFY_TIME AS RESOURCE_POOL_UPDATE_TIME, t1.MIGRATE_FROM_SVR_IP, t1.MIGRATE_FROM_SVR_PORT, t1.MANUAL_MIGRATE FROM oceanbase.DBA_OB_UNITS AS t1 JOIN oceanbase.DBA_OB_RESOURCE_POOLS AS t2 ON t1.RESOURCE_POOL_ID = t2.RESOURCE_POOL_ID JOIN oceanbase.DBA_OB_TENANTS t3 ON t2.TENANT_ID = t3.TENANT_ID WHERE 1=1 AND t2.tenant_id = ?, args: [1024] 2025-11-29 08:44:03.632 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.633 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state = 'ACTIVE' 2025-11-29 08:44:03.638 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.639 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2025-11-29 08:44:03.644 INFO 12717 --- [refresh-tenant-resource-cache-6,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:03.645 INFO 12717 --- [refresh-tenant-resource-cache-6,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.652 INFO 12717 --- [refresh-tenant-resource-cache-6,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(5000000) */ t1.UNIT_ID, t1.RESOURCE_POOL_ID, t1.UNIT_CONFIG_ID, t1.ZONE, t1.SVR_IP, t1.SVR_PORT, t1.STATUS, t2.REPLICA_TYPE, t2.TENANT_ID, t3.TENANT_NAME, t2.NAME AS RESOURCE_POOL_NAME, t2.MODIFY_TIME AS RESOURCE_POOL_UPDATE_TIME, t1.MIGRATE_FROM_SVR_IP, t1.MIGRATE_FROM_SVR_PORT, t1.MANUAL_MIGRATE FROM oceanbase.DBA_OB_UNITS AS t1 JOIN oceanbase.DBA_OB_RESOURCE_POOLS AS t2 ON t1.RESOURCE_POOL_ID = t2.RESOURCE_POOL_ID JOIN oceanbase.DBA_OB_TENANTS t3 ON t2.TENANT_ID = t3.TENANT_ID WHERE 1=1 AND t2.tenant_id = ?, args: [1022] 2025-11-29 08:44:03.675 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:03.675 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:03.675 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:03.675 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:03.675 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:03.675 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:03.676 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:03.676 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:03.676 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.677 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state = 'ACTIVE' 2025-11-29 08:44:03.698 INFO 12717 --- [ocp-async-35,ecdf4f272df1ff83,462e24924cafc5b9] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:03.858 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:03.858 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:03.858 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:03.858 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:03.858 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.859 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2025-11-29 08:44:03.936 INFO 12717 --- [print-time-series-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Async compute queue max size = 0, current size = 0. 2025-11-29 08:44:03.993 INFO 12717 --- [ocp-schedules-17,184bd89e0b7487f9,84b1297028b8ab61] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog instances, size=0 2025-11-29 08:44:03.994 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:03.995 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state = 'ACTIVE' 2025-11-29 08:44:04.041 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:04.041 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2025-11-29 08:44:04.176 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:04.177 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state = 'ACTIVE' 2025-11-29 08:44:04.182 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:04.182 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2025-11-29 08:44:04.185 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.p.o.i.OasSessionStatServiceImpl : begin to get deadlock of DB 2025-11-29 08:44:04.206 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=lhp6796, dbUser=root 2025-11-29 08:44:04.228 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:04.230 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY where report_time > ? and report_time < ?;, args: [2025-11-22 08:44:04, 2025-11-29 08:44:04] 2025-11-29 08:44:04.255 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=nextcloud, dbUser=root 2025-11-29 08:44:04.257 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:04.259 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY where report_time > ? and report_time < ?;, args: [2025-11-22 08:44:04, 2025-11-29 08:44:04] 2025-11-29 08:44:04.284 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=oms, dbUser=root 2025-11-29 08:44:04.286 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:04.287 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY where report_time > ? and report_time < ?;, args: [2025-11-22 08:44:04, 2025-11-29 08:44:04] 2025-11-29 08:44:04.313 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=pipeline, dbUser=root 2025-11-29 08:44:04.315 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:04.317 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY where report_time > ? and report_time < ?;, args: [2025-11-22 08:44:04, 2025-11-29 08:44:04] 2025-11-29 08:44:04.343 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=root 2025-11-29 08:44:04.344 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:04.344 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:04.344 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:04.344 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:04.344 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:04.344 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:04.344 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:04.344 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:04.345 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:04.346 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY where report_time > ? and report_time < ?;, args: [2025-11-22 08:44:04, 2025-11-29 08:44:04] 2025-11-29 08:44:04.370 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=wecom_bot, dbUser=root 2025-11-29 08:44:04.372 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:04.373 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY where report_time > ? and report_time < ?;, args: [2025-11-22 08:44:04, 2025-11-29 08:44:04] 2025-11-29 08:44:04.397 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=zto, dbUser=root 2025-11-29 08:44:04.398 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:04.400 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY where report_time > ? and report_time < ?;, args: [2025-11-22 08:44:04, 2025-11-29 08:44:04] 2025-11-29 08:44:04.403 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.p.o.i.OasSessionStatServiceImpl : finish fetching tenant session data of DB 2025-11-29 08:44:04.403 INFO 12717 --- [ForkJoinPool-7-worker-0,b6b8e753cfae423f,d5832545655bf6ae] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching tenant session stat of DB 2025-11-29 08:44:04.403 INFO 12717 --- [pool-11-thread-14,b6b8e753cfae423f,63ca90833ba460d8] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching session stat 2025-11-29 08:44:04.458 INFO 12717 --- [host-check-clockdiff9,95eb62dc9b2c413b,8c8172af3aafe5a7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764377044 -2 -2 , error= 2025-11-29 08:44:04.548 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:04.553 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:04.833 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,43d4aea13ede7446,63880a6c1e7a17ff] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=43d4aea13ede7446, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:04.833 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,43d4aea13ede7446,63880a6c1e7a17ff] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=43d4aea13ede7446, duration=2 ms] 2025-11-29 08:44:05.074 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,817d244fcf5ff53f,acaba4cc18f09a10] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=817d244fcf5ff53f, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:05.082 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,817d244fcf5ff53f,acaba4cc18f09a10] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=817d244fcf5ff53f, duration=9 ms] 2025-11-29 08:44:05.188 INFO 12717 --- [ocp-schedules-3,7beb3391f2d6fa45,9808927a3f325905] .o.o.s.t.b.c.AbstractShardingClusterTask : Collect compaction for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:44:05.407 INFO 12717 --- [prometheus-minute-schedule-1,1cda0c26c6d87c35,69e3e9823a5cca34] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:05.425 INFO 12717 --- [prometheus-minute-schedule-1,1cda0c26c6d87c35,69e3e9823a5cca34] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:05.466 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,c085d5228ed05af9,47fd4422c3fd0391] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=c085d5228ed05af9, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:05.466 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,c085d5228ed05af9,47fd4422c3fd0391] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=c085d5228ed05af9, duration=1 ms] 2025-11-29 08:44:05.647 WARN 12717 --- [host-check-clockdiff11,95eb62dc9b2c413b,927406fe65369564] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.48], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:05.647 ERROR 12717 --- [host-check-clockdiff11,95eb62dc9b2c413b,927406fe65369564] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.48] 2025-11-29 08:44:05.647 WARN 12717 --- [host-check-clockdiff11,95eb62dc9b2c413b,927406fe65369564] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=12, ip=10.202.22.48, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.48 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:05.699 INFO 12717 --- [ocp-async-14,f4fd02e84b1a4fc6,283dd0a038dcdac5] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:05.703 WARN 12717 --- [host-check-clockdiff8,95eb62dc9b2c413b,03be3893433b22db] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.60], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:05.703 ERROR 12717 --- [host-check-clockdiff8,95eb62dc9b2c413b,03be3893433b22db] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.60] 2025-11-29 08:44:05.703 WARN 12717 --- [host-check-clockdiff8,95eb62dc9b2c413b,03be3893433b22db] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=9, ip=10.202.22.60, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.60 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:06.963 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,6a22ea63fbb3311b,b782ceb020d96e59] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=6a22ea63fbb3311b, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:06.965 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,6a22ea63fbb3311b,b782ceb020d96e59] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=6a22ea63fbb3311b, duration=3 ms] 2025-11-29 08:44:07.127 INFO 12717 --- [ocp-schedules-24,359b501c98d47ea6,f051381976588c00] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-11-29 08:44:07.272 INFO 12717 --- [ocp-schedules-24,359b501c98d47ea6,f051381976588c00] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:44:07.272 INFO 12717 --- [ocp-schedules-24,359b501c98d47ea6,f051381976588c00] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:44:07.554 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:07.556 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:07.699 INFO 12717 --- [ocp-async-14,53e6e3db7465171f,38f566d484541c0e] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:08.369 INFO 12717 --- [ocp-schedules-22,2976bb2f1355fd66,a3123e70f93d74bb] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:44:08.373 INFO 12717 --- [ocp-schedules-22,2976bb2f1355fd66,a3123e70f93d74bb] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:44:08.407 INFO 12717 --- [prometheus-minute-schedule-1,51721aea4870e0c6,a22ed52853f941a3] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:08.419 INFO 12717 --- [prometheus-minute-schedule-1,51721aea4870e0c6,a22ed52853f941a3] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:09.696 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:44:09.695822+08:00, lastProcessTime=2025-11-29T08:43:59.702011+08:00 2025-11-29 08:44:09.699 INFO 12717 --- [ocp-async-33,bc8abb77c4147f34,34427746262c4411] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:09.866 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-77,4926f5942e7e64e4,b42dd2e4c3c7b7e2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=4926f5942e7e64e4, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:09.871 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-77,4926f5942e7e64e4,b42dd2e4c3c7b7e2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=4926f5942e7e64e4, duration=6 ms] 2025-11-29 08:44:10.019 INFO 12717 --- [ocp-schedules-4,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=3 2025-11-29 08:44:10.019 INFO 12717 --- [ocp-schedules-15,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=3 2025-11-29 08:44:10.101 INFO 12717 --- [Collect obproxy1,5b8509797072071d,011492ee46214ea6] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:44:10.101 INFO 12717 --- [Collect obproxy1,5b8509797072071d,011492ee46214ea6] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:10.102 INFO 12717 --- [Collect obproxy2,70f1aef4b76b6d3a,a71e7f8b9682f191] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:44:10.102 INFO 12717 --- [Check obproxy3,b0d2dce32d050e58,88030e5e1090b269] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:44:10.102 INFO 12717 --- [Check obproxy2,dcee14700a25980b,290d2d79ebb362f9] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:44:10.102 INFO 12717 --- [Collect obproxy2,70f1aef4b76b6d3a,a71e7f8b9682f191] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:10.102 INFO 12717 --- [Check obproxy3,b0d2dce32d050e58,88030e5e1090b269] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:10.102 INFO 12717 --- [Check obproxy2,dcee14700a25980b,290d2d79ebb362f9] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:10.102 INFO 12717 --- [Collect obproxy3,b1e3a75665188ea3,603180ce69da67e2] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:44:10.102 INFO 12717 --- [Collect obproxy3,b1e3a75665188ea3,603180ce69da67e2] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:10.120 INFO 12717 --- [Check obproxy1,be131ef1e14cfe02,46831443cd7e7bce] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:44:10.120 INFO 12717 --- [Check obproxy1,be131ef1e14cfe02,46831443cd7e7bce] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:10.189 INFO 12717 --- [Collect obproxy3,b1e3a75665188ea3,603180ce69da67e2] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:44:10.193 INFO 12717 --- [Collect obproxy3,b1e3a75665188ea3,603180ce69da67e2] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:44:10.196 INFO 12717 --- [Collect obproxy3,b1e3a75665188ea3,603180ce69da67e2] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:44:10.197 INFO 12717 --- [Check obproxy3,b0d2dce32d050e58,88030e5e1090b269] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:44:10.202 INFO 12717 --- [Check obproxy3,b0d2dce32d050e58,88030e5e1090b269] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2025-11-29 08:44:10.202 INFO 12717 --- [Check obproxy3,b0d2dce32d050e58,88030e5e1090b269] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 3 proxy_id: 3, need to sync to meta: false 2025-11-29 08:44:10.210 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:44:09.695822+08:00 2025-11-29 08:44:10.225 INFO 12717 --- [Collect obproxy2,70f1aef4b76b6d3a,a71e7f8b9682f191] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:44:10.226 INFO 12717 --- [Check obproxy2,dcee14700a25980b,290d2d79ebb362f9] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:44:10.230 INFO 12717 --- [Check obproxy2,dcee14700a25980b,290d2d79ebb362f9] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-11-29 08:44:10.230 INFO 12717 --- [Check obproxy2,dcee14700a25980b,290d2d79ebb362f9] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 2, need to sync to meta: false 2025-11-29 08:44:10.231 INFO 12717 --- [Collect obproxy2,70f1aef4b76b6d3a,a71e7f8b9682f191] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:44:10.236 INFO 12717 --- [Collect obproxy2,70f1aef4b76b6d3a,a71e7f8b9682f191] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:44:10.238 INFO 12717 --- [Collect obproxy1,5b8509797072071d,011492ee46214ea6] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:44:10.240 INFO 12717 --- [Collect obproxy1,5b8509797072071d,011492ee46214ea6] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:44:10.242 INFO 12717 --- [Check obproxy1,be131ef1e14cfe02,46831443cd7e7bce] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:44:10.244 INFO 12717 --- [Collect obproxy1,5b8509797072071d,011492ee46214ea6] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:44:10.246 INFO 12717 --- [Check obproxy1,be131ef1e14cfe02,46831443cd7e7bce] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-11-29 08:44:10.246 INFO 12717 --- [Check obproxy1,be131ef1e14cfe02,46831443cd7e7bce] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 1, need to sync to meta: false 2025-11-29 08:44:10.271 INFO 12717 --- [ocp-schedules-18,5be69d6e089440ca,efc7029dba16f0f4] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:44:10.277 INFO 12717 --- [ocp-schedules-13,703313ae15058e73,a88d4624f2703b55] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=2, successCount=2 2025-11-29 08:44:10.355 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-100,d64cd90a3eb0b67a,5a58fd845b4fd9d2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=d64cd90a3eb0b67a, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:10.356 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-100,d64cd90a3eb0b67a,5a58fd845b4fd9d2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=d64cd90a3eb0b67a, duration=2 ms] 2025-11-29 08:44:10.470 INFO 12717 --- [ocp-schedules-17,f8b7c30adeb2e694,e374777e7cd7a9d1] c.o.o.s.common.scheduled.PerfSchedules : Collect trans stat for alarm, time=2025-11-29T08:44:10.470+08:00, clusters=[1, 2] 2025-11-29 08:44:10.471 INFO 12717 --- [ocp-schedules-19,00a964e83174903e,5c05906fb7ab685f] c.o.o.s.common.scheduled.HostSchedules : Check host version schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:44:10.471 INFO 12717 --- [ocp-schedules-19,00a964e83174903e,5c05906fb7ab685f] o.o.c.t.e.u.DistributedExclusiveExecutor : Empty item list, taskName=host-check-agent-version, skip execute 2025-11-29 08:44:10.479 INFO 12717 --- [ocp-schedules-17,f8b7c30adeb2e694,e374777e7cd7a9d1] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster ocpserver, trans stat result is empty. 2025-11-29 08:44:10.488 INFO 12717 --- [ocp-schedules-17,f8b7c30adeb2e694,e374777e7cd7a9d1] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster DB, trans stat result is empty. 2025-11-29 08:44:10.556 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:10.561 INFO 12717 --- [ocp-schedules-23,2ad21c79e41a5b4d,fcf65e27f0e9fbad] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:44:10.563 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:10.648 INFO 12717 --- [ocp-schedules-14,33039a1749d759d1,275ca2366d57da58] .o.o.s.t.b.c.AbstractShardingClusterTask : Check cluster cgroup value. executed completed, allCount=2, successCount=2 2025-11-29 08:44:10.972 INFO 12717 --- [async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 28, ruleSize = 28. 2025-11-29 08:44:11.407 INFO 12717 --- [prometheus-minute-schedule-1,19b3d64ec543ae81,10fb0a252c09ca11] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:11.421 INFO 12717 --- [prometheus-minute-schedule-1,19b3d64ec543ae81,10fb0a252c09ca11] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:11.808 INFO 12717 --- [ocp-async-33,3c2e317312c82031,b6cfab7f5a9752a3] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:13.802 INFO 12717 --- [ocp-schedules-29,97384e8339d99067,24e83c85d8e658e8] c.o.o.s.c.scheduled.TenantSchedules : Check all tenant service name conflict 2025-11-29 08:44:13.802 INFO 12717 --- [ocp-schedules-29,97384e8339d99067,24e83c85d8e658e8] .t.CheckAllTenantServiceNameConflictTask : Check all tenant service name conflict. 2025-11-29 08:44:13.802 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:13.803 INFO 12717 --- [ocp-schedules-27,ffabd8ec15831530,47b2f9eb1844d7dc] c.o.o.s.common.scheduled.OcpSchedules : Cleanup ocp connection pools. 2025-11-29 08:44:13.803 INFO 12717 --- [ocp-schedules-27,ffabd8ec15831530,47b2f9eb1844d7dc] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] clean up the expired connectors. 2025-11-29 08:44:13.803 INFO 12717 --- [ocp-schedules-2,264a6f62b1aeca64,c12cf2c520072106] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:44:13.805 INFO 12717 --- [ocp-async-6,5d7aede4bad046b9,aeeb109f2c8456cb] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:44:13.809 INFO 12717 --- [ocp-async-22,361d48a7702aff31,85418ec70fce91c0] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:13.810 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-98,e0d8a58886f2fd88,3d11247c28d69cde] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=e0d8a58886f2fd88, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:13.810 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-98,e0d8a58886f2fd88,3d11247c28d69cde] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=e0d8a58886f2fd88, duration=3 ms] 2025-11-29 08:44:13.811 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:13.819 INFO 12717 --- [alarm-detect-17,264a6f62b1aeca64,22d63491c7a4b760] c.o.o.alarm.core.detect.MetricDetector : alarmType host_agent_res_memory_over_threshold detect time range 1764376983 ~ 1764377043 matched, alarm-level: 3, evaluate result MultiExpressionEvaluateResult(metricSnapshots={host_agent_resident_memory_gb=[SeriesMetricValues(metric=Metric{name='host_agent_resident_memory_gb', labels={app=HOST, host=10.202.22.52, host_name=localhost.localdomain, ob_cluster_id=1743215594, ob_cluster_name=DB, obzone=, process=ocp_monagent, svr_ip=10.202.22.52}}, data=[ValueData(timestamp=1764377040, value=1.633)])]}, metricEvaluateResults={host_agent_resident_memory_gb > 1.5=[ExpressionEvaluateResult(matchValue=1.5, metric=host_agent_resident_memory_gb, match=true, value=1.633, values={1764377040=1.633}, valueCount=1, labels={app=HOST, host=10.202.22.52, host_name=localhost.localdomain, ob_cluster_id=1743215594, ob_cluster_name=DB, obzone=, process=ocp_monagent, svr_ip=10.202.22.52})]}, matched=true, isContainValidValue=null, message=null, level=3) 2025-11-29 08:44:13.869 WARN 12717 --- [scanner-11213817656-1,264a6f62b1aeca64,ef4afd3cb860ec6a] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:44:15.165 INFO 12717 --- [prometheus-minute-schedule-1,3825e746f8964cdb,76f24580131fcc17] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:15.175 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,e20422572151195b,a2403f525d495e2c] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=e20422572151195b, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:15.176 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,e20422572151195b,a2403f525d495e2c] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=e20422572151195b, duration=3 ms] 2025-11-29 08:44:15.200 INFO 12717 --- [prometheus-minute-schedule-1,3825e746f8964cdb,76f24580131fcc17] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:15.359 INFO 12717 --- [metric-archive-second-3,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=200, affectRows=200 2025-11-29 08:44:15.366 INFO 12717 --- [metric-archive-second-3,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=200, validCount=199 2025-11-29 08:44:17.134 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:17.138 INFO 12717 --- [ocp-async-33,f1b2b22a9f11464d,866727014a0bcc12] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:17.145 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,b6d3842138e23aac,6d6a22feaeabfbd2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=b6d3842138e23aac, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:17.145 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,b26d903391a92f10,7f669d3898872abf] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=b26d903391a92f10, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:17.150 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:17.164 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-48,b26d903391a92f10,7f669d3898872abf] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=b26d903391a92f10, duration=20 ms] 2025-11-29 08:44:17.165 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,b6d3842138e23aac,6d6a22feaeabfbd2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=b6d3842138e23aac, duration=21 ms] 2025-11-29 08:44:17.168 ERROR 12717 --- [metric-archive-async-12,,] c.o.o.m.storage.AsyncComputeDataDao : Archive node list to table metric_data_async error, e = Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: monitor-connect-pool - Connection is not available, request timed out after 2000ms. 2025-11-29 08:44:17.168 ERROR 12717 --- [metric-archive-async-13,,] c.o.o.m.storage.AsyncComputeDataDao : Archive node list to table metric_data_async error, e = Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: monitor-connect-pool - Connection is not available, request timed out after 2000ms. 2025-11-29 08:44:17.168 ERROR 12717 --- [metric-archive-async-17,,] c.o.o.m.storage.AsyncComputeDataDao : Archive node list to table metric_data_async error, e = Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: monitor-connect-pool - Connection is not available, request timed out after 2000ms. 2025-11-29 08:44:17.168 ERROR 12717 --- [metric-archive-async-16,,] c.o.o.m.storage.AsyncComputeDataDao : Archive node list to table metric_data_async error, e = Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: monitor-connect-pool - Connection is not available, request timed out after 2000ms. 2025-11-29 08:44:17.204 INFO 12717 --- [metric-archive-second-13,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=400, affectRows=400 2025-11-29 08:44:17.211 INFO 12717 --- [metric-archive-second-13,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=397, validCount=397 2025-11-29 08:44:17.258 INFO 12717 --- [metric-archive-second-2,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=351, affectRows=351 2025-11-29 08:44:17.264 INFO 12717 --- [metric-archive-second-2,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=344, validCount=342 2025-11-29 08:44:17.290 INFO 12717 --- [metric-archive-second-15,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=1000, affectRows=1000 2025-11-29 08:44:17.302 INFO 12717 --- [metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=285, affectRows=285 2025-11-29 08:44:17.302 INFO 12717 --- [metric-archive-second-15,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=997, validCount=996 2025-11-29 08:44:17.305 INFO 12717 --- [metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=285, validCount=282 2025-11-29 08:44:17.407 INFO 12717 --- [prometheus-minute-schedule-1,d63e3cf81ef92fbc,9a8bc3be053c8786] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:17.420 INFO 12717 --- [prometheus-minute-schedule-1,d63e3cf81ef92fbc,9a8bc3be053c8786] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:17.527 INFO 12717 --- [metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=379, affectRows=379 2025-11-29 08:44:17.529 INFO 12717 --- [metric-archive-second-20,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=1000, affectRows=1000 2025-11-29 08:44:17.530 INFO 12717 --- [metric-archive-second-9,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=430, affectRows=430 2025-11-29 08:44:17.531 INFO 12717 --- [metric-archive-async-27,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1764376980, nodeCount=10, affectRows=10 2025-11-29 08:44:17.531 INFO 12717 --- [metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=301, affectRows=301 2025-11-29 08:44:17.531 INFO 12717 --- [metric-archive-second-17,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=366, affectRows=366 2025-11-29 08:44:17.532 INFO 12717 --- [metric-archive-second-12,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=1000, affectRows=1000 2025-11-29 08:44:17.532 INFO 12717 --- [metric-archive-async-31,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1764376980, nodeCount=89, affectRows=89 2025-11-29 08:44:17.533 INFO 12717 --- [metric-archive-second-19,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=474, affectRows=474 2025-11-29 08:44:17.533 INFO 12717 --- [metric-archive-async-19,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1764376980, nodeCount=7, affectRows=7 2025-11-29 08:44:17.535 INFO 12717 --- [metric-archive-second-9,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=425, validCount=423 2025-11-29 08:44:17.535 INFO 12717 --- [metric-archive-second-14,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=1000, affectRows=1000 2025-11-29 08:44:17.535 INFO 12717 --- [metric-archive-second-17,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=356, validCount=356 2025-11-29 08:44:17.536 INFO 12717 --- [metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=298, validCount=297 2025-11-29 08:44:17.537 INFO 12717 --- [metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=379, validCount=379 2025-11-29 08:44:17.539 INFO 12717 --- [metric-archive-second-20,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=994, validCount=991 2025-11-29 08:44:17.540 INFO 12717 --- [metric-archive-second-19,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=472, validCount=471 2025-11-29 08:44:17.542 INFO 12717 --- [metric-archive-second-12,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=990, validCount=985 2025-11-29 08:44:17.545 INFO 12717 --- [metric-archive-second-14,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=992, validCount=992 2025-11-29 08:44:17.549 INFO 12717 --- [metric-archive-second-10,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=1000, affectRows=1000 2025-11-29 08:44:17.559 INFO 12717 --- [metric-archive-second-10,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=989, validCount=984 2025-11-29 08:44:18.315 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-89,27b7d61aefbe4d6d,86950de659d6b7bd] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=27b7d61aefbe4d6d, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:18.315 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-89,27b7d61aefbe4d6d,86950de659d6b7bd] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=27b7d61aefbe4d6d, duration=1 ms] 2025-11-29 08:44:18.327 INFO 12717 --- [ocp-schedules-11,a147cace3efd1723,ea26ef4f378a1a9e] c.o.o.a.service.OcpAlarmInspectService : delete inactive alarm event count 0 2025-11-29 08:44:18.327 INFO 12717 --- [ocp-schedules-11,a147cace3efd1723,ea26ef4f378a1a9e] c.o.o.a.service.OcpAlarmInspectService : deleteInactiveAlarmEvents count 0 2025-11-29 08:44:18.339 INFO 12717 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.340 INFO 12717 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:18.348 INFO 12717 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.349 INFO 12717 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:18.350 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule start 2025-11-29 08:44:18.401 INFO 12717 --- [ocp-schedules-5,38f261e75b73fac1,614c253116bdf346] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:44:18.404 INFO 12717 --- [metric-archive-second-8,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=931, affectRows=931 2025-11-29 08:44:18.421 INFO 12717 --- [metric-archive-second-8,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=927, validCount=926 2025-11-29 08:44:18.427 INFO 12717 --- [ocp-schedules-5,38f261e75b73fac1,614c253116bdf346] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:44:18.453 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=1, name=ocpserver, obClusterId=1743215593, status=RUNNING} 2025-11-29 08:44:18.476 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING} 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-8) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-5) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-2) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-4) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-1) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-11) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-10) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-7) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-9) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (ocpserver:1743215593-1) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-6) 2025-11-29 08:44:18.497 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (DB:1743215594-3) 2025-11-29 08:44:18.508 INFO 12717 --- [sql-rollup-worker-34,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=10, level=0->1, c=0) - not enough cached interval (source=2025-11-29T08:42:09, target=2025-11-29T08:40:09, granule=120000ms, delay=120000ms) 2025-11-29 08:44:18.509 INFO 12717 --- [sql-rollup-worker-34,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=10, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:40:09, target=2025-11-29T08:36:09, granule=600000ms, delay=240000ms) 2025-11-29 08:44:18.509 INFO 12717 --- [sql-rollup-worker-31,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=8, level=0->1, c=0) - not enough cached interval (source=2025-11-29T08:41:47, target=2025-11-29T08:39:47, granule=120000ms, delay=120000ms) 2025-11-29 08:44:18.509 INFO 12717 --- [sql-rollup-worker-31,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=8, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:39:47, target=2025-11-29T08:29:17, granule=600000ms, delay=630000ms) 2025-11-29 08:44:18.510 INFO 12717 --- [sql-rollup-worker-21,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=1, level=0->1, c=0) - not enough cached interval (source=2025-11-29T08:42:10, target=2025-11-29T08:40:09, granule=120000ms, delay=121000ms) 2025-11-29 08:44:18.510 INFO 12717 --- [sql-rollup-worker-21,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=1, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:40:09, target=2025-11-29T08:29:09, granule=600000ms, delay=660000ms) 2025-11-29 08:44:18.510 INFO 12717 --- [sql-rollup-worker-17,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.511 INFO 12717 --- [sql-rollup-worker-19,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.511 INFO 12717 --- [sql-rollup-worker-26,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.511 INFO 12717 --- [sql-rollup-worker-30,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.511 INFO 12717 --- [sql-rollup-worker-29,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.511 INFO 12717 --- [sql-rollup-worker-35,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.511 INFO 12717 --- [sql-rollup-worker-33,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.511 INFO 12717 --- [sql-rollup-worker-32,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.512 INFO 12717 --- [sql-rollup-worker-19,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:18.512 INFO 12717 --- [sql-rollup-worker-30,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:18.512 INFO 12717 --- [sql-rollup-worker-17,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:18.513 INFO 12717 --- [sql-rollup-worker-35,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:18.513 INFO 12717 --- [sql-rollup-worker-19,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.513 INFO 12717 --- [sql-rollup-worker-29,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:18.513 INFO 12717 --- [sql-rollup-worker-26,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:18.514 INFO 12717 --- [sql-rollup-worker-30,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.514 INFO 12717 --- [sql-rollup-worker-17,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.514 INFO 12717 --- [sql-rollup-worker-33,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:18.514 INFO 12717 --- [sql-rollup-worker-32,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:18.514 INFO 12717 --- [sql-rollup-worker-19,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:44:18.515 INFO 12717 --- [sql-rollup-worker-35,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.515 INFO 12717 --- [sql-rollup-worker-26,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.515 INFO 12717 --- [sql-rollup-worker-30,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:44:18.515 INFO 12717 --- [sql-rollup-worker-17,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:44:18.515 INFO 12717 --- [sql-rollup-worker-29,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.516 INFO 12717 --- [sql-rollup-worker-32,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.516 INFO 12717 --- [sql-rollup-worker-33,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:18.516 INFO 12717 --- [sql-rollup-worker-35,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:44:18.517 INFO 12717 --- [sql-rollup-worker-26,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:44:18.517 INFO 12717 --- [sql-rollup-worker-29,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:44:18.517 INFO 12717 --- [sql-rollup-worker-33,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:44:18.517 INFO 12717 --- [sql-rollup-worker-17,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup newly (cluster=ocpserver:1743215593, server=1, level=0->1, c=0) - not enough cached interval (source=2025-03-29T10:42:31, earliest=2025-11-28T08:44:18, granule=120000ms, delay=-21074507000ms) 2025-11-29 08:44:18.517 INFO 12717 --- [sql-rollup-worker-17,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup greedily (cluster=ocpserver:1743215593, server=1, level=1->2, c=0) - source level not data, skipped 2025-11-29 08:44:18.517 INFO 12717 --- [sql-rollup-worker-32,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:44:20.068 INFO 12717 --- [ocp-async-6,fe75c50f8a2fb902,32fec7084a28b419] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:20.079 INFO 12717 --- [sql-rollup-worker-32,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] slow query, durationMillis=1562, sql=select now() 2025-11-29 08:44:20.086 INFO 12717 --- [ocp-schedules-4,e3ed24d5a08ee266,bfeddcfffd415d07] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-11-29 08:44:20.088 WARN 12717 --- [ocp-updater-0,,] c.o.o.c.d.service.DefaultServiceUpdater : Deleted expired service: DistributedService{name=SQL_INDEX_DIAGNOSER, serverId=10.202.22.58:8080, expireInSeconds=30, createTime=2025-11-29T08:38:19+08:00, updateTime=2025-11-29T08:43:49+08:00, expireTime=null, isLeader=false} 2025-11-29 08:44:20.092 WARN 12717 --- [ocp-updater-0,,] c.o.o.c.d.service.DefaultServiceUpdater : No leader or more than one leader exists for service SQL_INDEX_DIAGNOSER, Incorrect result size: expected 1, actual 0 2025-11-29 08:44:20.104 ERROR 12717 --- [sql-rollup-worker-26,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup server-side (cluster=DB:1743215594, server=3, level=0->1, c=0) - source level expect data, actual none (from=2025-11-29T08:39:03) 2025-11-29 08:44:20.109 INFO 12717 --- [metric-archive-second-24,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1764376980, nodeCount=971, affectRows=971 2025-11-29 08:44:20.121 INFO 12717 --- [metric-archive-second-24,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1764376980, nodeCount=969, validCount=967 2025-11-29 08:44:20.151 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:20.152 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:20.162 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:44:20.161395+08:00, lastProcessTime=2025-11-29T08:44:09.695822+08:00 2025-11-29 08:44:20.216 INFO 12717 --- [sql-rollup-worker-24,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:20.216 INFO 12717 --- [sql-rollup-worker-24,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-11-29 08:44:20.218 INFO 12717 --- [sql-rollup-worker-24,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:20.218 INFO 12717 --- [sql-rollup-worker-24,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2025-11-29 08:44:20.284 INFO 12717 --- [ocp-schedules-6,45a41b1190d39a2c,23a9f048e9262b19] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:44:20.357 INFO 12717 --- [sql-rollup-worker-30,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=2, level=0->1, c=0) - source=2025-11-29T08:42:07, start=2025-11-29T08:38:06, end=2025-11-29T08:40:36, updates=173, granule=120000ms, delay=241000ms 2025-11-29 08:44:20.358 INFO 12717 --- [sql-rollup-worker-24,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=5, level=0->1, c=0) - source=2025-11-29T08:41:54, start=2025-11-29T08:38:24, end=2025-11-29T08:40:54, updates=77, granule=120000ms, delay=210000ms 2025-11-29 08:44:20.358 INFO 12717 --- [sql-rollup-worker-33,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=4, level=0->1, c=0) - source=2025-11-29T08:41:57, start=2025-11-29T08:37:57, end=2025-11-29T08:40:27, updates=138, granule=120000ms, delay=240000ms 2025-11-29 08:44:20.359 INFO 12717 --- [sql-rollup-worker-19,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=7, level=0->1, c=0) - source=2025-11-29T08:41:55, start=2025-11-29T08:37:56, end=2025-11-29T08:40:26, updates=120, granule=120000ms, delay=239000ms 2025-11-29 08:44:20.359 INFO 12717 --- [sql-rollup-worker-26,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=3, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:42:02, target=2025-11-29T08:41:03, granule=120000ms, delay=59000ms) 2025-11-29 08:44:20.359 INFO 12717 --- [sql-rollup-worker-26,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=3, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:41:03, target=2025-11-29T08:30:33, granule=600000ms, delay=630000ms) 2025-11-29 08:44:20.359 INFO 12717 --- [sql-rollup-worker-29,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=11, level=0->1, c=0) - source=2025-11-29T08:41:55, start=2025-11-29T08:39:25, end=2025-11-29T08:41:55, updates=78, granule=120000ms, delay=150000ms 2025-11-29 08:44:20.359 INFO 12717 --- [sql-rollup-worker-32,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=6, level=0->1, c=0) - source=2025-11-29T08:42:16, start=2025-11-29T08:38:16, end=2025-11-29T08:40:46, updates=14, granule=120000ms, delay=240000ms 2025-11-29 08:44:20.360 INFO 12717 --- [sql-rollup-worker-35,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=9, level=0->1, c=0) - source=2025-11-29T08:42:11, start=2025-11-29T08:39:41, end=2025-11-29T08:42:11, updates=81, granule=120000ms, delay=150000ms 2025-11-29 08:44:20.373 INFO 12717 --- [sql-rollup-worker-30,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=2, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:42:07, target=2025-11-29T08:40:07, granule=120000ms, delay=120000ms) 2025-11-29 08:44:20.373 INFO 12717 --- [sql-rollup-worker-32,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=6, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:42:16, target=2025-11-29T08:40:46, granule=120000ms, delay=90000ms) 2025-11-29 08:44:20.373 INFO 12717 --- [sql-rollup-worker-33,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=4, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:41:57, target=2025-11-29T08:40:27, granule=120000ms, delay=90000ms) 2025-11-29 08:44:20.373 INFO 12717 --- [sql-rollup-worker-33,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=4, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:40:27, target=2025-11-29T08:30:57, granule=600000ms, delay=570000ms) 2025-11-29 08:44:20.373 INFO 12717 --- [sql-rollup-worker-32,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=6, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:40:46, target=2025-11-29T08:28:16, granule=600000ms, delay=750000ms) 2025-11-29 08:44:20.374 INFO 12717 --- [sql-rollup-worker-24,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=5, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:41:54, target=2025-11-29T08:40:24, granule=120000ms, delay=90000ms) 2025-11-29 08:44:20.374 INFO 12717 --- [sql-rollup-worker-19,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=7, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:41:55, target=2025-11-29T08:40:25, granule=120000ms, delay=90000ms) 2025-11-29 08:44:20.374 INFO 12717 --- [sql-rollup-worker-24,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=5, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:40:24, target=2025-11-29T08:28:54, granule=600000ms, delay=690000ms) 2025-11-29 08:44:20.374 INFO 12717 --- [sql-rollup-worker-19,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=7, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:40:25, target=2025-11-29T08:28:25, granule=600000ms, delay=720000ms) 2025-11-29 08:44:20.375 INFO 12717 --- [sql-rollup-worker-29,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=11, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:41:55, target=2025-11-29T08:41:55, granule=120000ms, delay=0ms) 2025-11-29 08:44:20.375 INFO 12717 --- [sql-rollup-worker-35,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=9, level=0->1, c=1) - not enough cached interval (source=2025-11-29T08:42:11, target=2025-11-29T08:42:11, granule=120000ms, delay=0ms) 2025-11-29 08:44:20.375 INFO 12717 --- [sql-rollup-worker-35,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=9, level=1->2, c=0) - not enough cached interval (source=2025-11-29T08:42:11, target=2025-11-29T08:29:10, granule=600000ms, delay=781000ms) 2025-11-29 08:44:20.407 INFO 12717 --- [prometheus-minute-schedule-1,a155b162acf79f12,a72205047dfc3a17] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:20.425 INFO 12717 --- [prometheus-minute-schedule-1,a155b162acf79f12,a72205047dfc3a17] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:20.473 INFO 12717 --- [sql-rollup-worker-29,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=11, level=1->2, c=0) - source=2025-11-29T08:41:55, start=2025-11-29T08:27:25, end=2025-11-29T08:37:25, updates=79, granule=600000ms, delay=870000ms 2025-11-29 08:44:20.481 INFO 12717 --- [sql-rollup-worker-29,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=11, level=1->2, c=1) - not enough cached interval (source=2025-11-29T08:41:55, target=2025-11-29T08:37:25, granule=600000ms, delay=270000ms) 2025-11-29 08:44:20.503 INFO 12717 --- [sql-rollup-worker-30,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=DB:1743215594, server=2, level=1->2, c=0) - source=2025-11-29T08:40:07, start=2025-11-29T08:24:37, end=2025-11-29T08:34:37, updates=213, granule=600000ms, delay=930000ms 2025-11-29 08:44:22.090 INFO 12717 --- [ocp-async-21,4c2f26dfabdbe3eb,1904072fd3fe0791] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:22.091 INFO 12717 --- [sql-rollup-worker-30,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=DB:1743215594, server=2, level=1->2, c=1) - not enough cached interval (source=2025-11-29T08:40:07, target=2025-11-29T08:34:37, granule=600000ms, delay=330000ms) 2025-11-29 08:44:22.091 INFO 12717 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule end, elapsed=3741ms, period=120000ms 2025-11-29 08:44:22.102 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:44:20.161395+08:00 2025-11-29 08:44:22.111 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-91,48f60660f06aada5,56587ba54e8474a9] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=48f60660f06aada5, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:22.111 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-91,48f60660f06aada5,56587ba54e8474a9] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=48f60660f06aada5, duration=3 ms] 2025-11-29 08:44:24.068 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:24.069 INFO 12717 --- [prometheus-minute-schedule-1,8b9099ea2c25dc69,3ae24319c160fa73] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:24.070 INFO 12717 --- [ocp-schedules-21,7b38f3e98da1e43e,fce4fd940d1702ad] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:44:24.070 INFO 12717 --- [ocp-async-34,e19c200edbdfea26,2ddeb3fc8b89cc92] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:44:24.076 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-87,1e1f668ae6f1f06b,85e6b948fc1eb3fb] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=1e1f668ae6f1f06b, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:24.076 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-87,1e1f668ae6f1f06b,85e6b948fc1eb3fb] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=1e1f668ae6f1f06b, duration=2 ms] 2025-11-29 08:44:24.083 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:24.084 INFO 12717 --- [ocp-async-23,e31346d2f90a50aa,5bbc7b634d94d7b9] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:24.110 INFO 12717 --- [prometheus-minute-schedule-1,8b9099ea2c25dc69,3ae24319c160fa73] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:24.149 WARN 12717 --- [scanner-11213817880-1,7b38f3e98da1e43e,be96f33e304b8822] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:44:24.194 INFO 12717 --- [alarm-detect-15,7b38f3e98da1e43e,550077338f498081] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008342, data is empty. 2025-11-29 08:44:24.199 INFO 12717 --- [alarm-detect-19,7b38f3e98da1e43e,b4f7e18fbbb51dfd] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008342, data is empty. 2025-11-29 08:44:24.206 INFO 12717 --- [alarm-detect-7,7b38f3e98da1e43e,9ebfe604f5fcff67] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008342, data is empty. 2025-11-29 08:44:24.215 INFO 12717 --- [alarm-detect-15,7b38f3e98da1e43e,550077338f498081] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008342, data is empty. 2025-11-29 08:44:24.227 INFO 12717 --- [alarm-detect-19,7b38f3e98da1e43e,b4f7e18fbbb51dfd] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008342, data is empty. 2025-11-29 08:44:24.231 INFO 12717 --- [alarm-detect-7,7b38f3e98da1e43e,9ebfe604f5fcff67] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008342, data is empty. 2025-11-29 08:44:24.265 INFO 12717 --- [metric-archive-async-17,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1764376980, nodeCount=107, affectRows=107 2025-11-29 08:44:24.321 INFO 12717 --- [alarm-detect-11,7b38f3e98da1e43e,e08fa7230ac5ec08] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008342, data is empty. 2025-11-29 08:44:24.330 INFO 12717 --- [alarm-detect-11,7b38f3e98da1e43e,e08fa7230ac5ec08] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008342, data is empty. 2025-11-29 08:44:26.133 INFO 12717 --- [ocp-schedules-22,2e16e2c082a900a5,632356d275900fc2] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info 2025-11-29 08:44:26.136 INFO 12717 --- [ocp-async-1,20212abaa79b4197,78b97c9653c22680] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:26.140 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,74f7ec0200aa73a9,7592684d24e181f5] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=74f7ec0200aa73a9, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:26.141 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-101,74f7ec0200aa73a9,7592684d24e181f5] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=74f7ec0200aa73a9, duration=3 ms] 2025-11-29 08:44:26.165 INFO 12717 --- [ocp-schedules-22,2e16e2c082a900a5,632356d275900fc2] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Invalidate clusters [] cache info success. 2025-11-29 08:44:26.165 INFO 12717 --- [ocp-schedules-22,2e16e2c082a900a5,632356d275900fc2] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info success, cached size: 2 2025-11-29 08:44:26.228 INFO 12717 --- [ocp-schedules-8,c0135be0ed998dc6,6f4cddddf19fdab4] c.o.o.s.common.scheduled.PerfSchedules : Oas sql curing, clusters=[] 2025-11-29 08:44:26.407 INFO 12717 --- [prometheus-minute-schedule-1,2d06a08b23fb4c9a,0c73be00e606e4f1] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:26.418 INFO 12717 --- [prometheus-minute-schedule-1,2d06a08b23fb4c9a,0c73be00e606e4f1] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:28.131 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:28.139 INFO 12717 --- [ocp-async-37,84cd60c2056fb0c7,cc1dfbc93b0ce4bc] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:28.141 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,1fc35ba60053b9b1,77be44ed4eb128e2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=1fc35ba60053b9b1, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:28.159 INFO 12717 --- [ocp-schedules-16,6de1fa233d3c3615,a555ff037e7eb221] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-11-29 08:44:28.161 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:28.164 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,1fc35ba60053b9b1,77be44ed4eb128e2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=1fc35ba60053b9b1, duration=24 ms] 2025-11-29 08:44:28.251 ERROR 12717 --- [metric-archive-async-12,,] c.o.o.m.service.OcpMetricArchiveService : Archive second nodes failed. org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [REPLACE INTO `metric_data_async` (`record_id`, `timestamp`, `data`, `interval`,`measurement_id`) VALUES (?, ?, ?, ?, ?)]; SQL state [HY000]; error code [4012]; (conn=1933625452) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again.; nested exception is java.sql.BatchUpdateException: (conn=1933625452) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1578) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:669) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:693) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:1036) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:1090) at com.oceanbase.ocp.monitor.storage.AsyncComputeDataDao.write(AsyncComputeDataDao.java:66) at com.oceanbase.ocp.monitor.storage.AsyncComputeDataDao$$FastClassBySpringCGLIB$$87cf8d28.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707) at com.oceanbase.ocp.monitor.storage.AsyncComputeDataDao$$EnhancerBySpringCGLIB$$ed28667e.write() at com.oceanbase.ocp.monitor.service.OcpMetricArchiveService.lambda$archiveAsyncCompute$5(OcpMetricArchiveService.java:231) at com.oceanbase.ocp.common.meter.MeterUtils.time(MeterUtils.java:66) at com.oceanbase.ocp.common.meter.DistributionSummaryGroup.time(DistributionSummaryGroup.java:184) at com.oceanbase.ocp.monitor.service.OcpMetricArchiveService.archiveAsyncCompute(OcpMetricArchiveService.java:222) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.sql.BatchUpdateException: (conn=1933625452) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.OceanBaseStatement.executeBatchExceptionEpilogue(OceanBaseStatement.java:450) at com.oceanbase.jdbc.OceanBaseStatement.executeBatchExceptionEpilogue(OceanBaseStatement.java:453) at com.oceanbase.jdbc.JDBC4PreparedStatement.executeBatch(JDBC4PreparedStatement.java:346) at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:127) at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java) at org.springframework.jdbc.core.JdbcTemplate.lambda$batchUpdate$4(JdbcTemplate.java:1050) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:650) ... 25 common frames omitted Caused by: java.sql.SQLTransientConnectionException: (conn=1933625452) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:128) at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:202) at com.oceanbase.jdbc.OceanBaseStatement.executeBatchExceptionEpilogue(OceanBaseStatement.java:446) ... 31 common frames omitted Caused by: com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException: Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException.of(OceanBaseSqlException.java:79) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeBatchRewrite(AbstractQueryProtocol.java:1252) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeBatchClient(AbstractQueryProtocol.java:704) at com.oceanbase.jdbc.JDBC4PreparedStatement.executeInternalBatch(JDBC4PreparedStatement.java:461) at com.oceanbase.jdbc.JDBC4PreparedStatement.executeBatch(JDBC4PreparedStatement.java:332) ... 29 common frames omitted Caused by: java.sql.SQLException: Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.readErrorPacket(AbstractQueryProtocol.java:2366) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.readPacket(AbstractQueryProtocol.java:2229) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:2119) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeBatchRewrite(AbstractQueryProtocol.java:1244) ... 32 common frames omitted 2025-11-29 08:44:28.251 ERROR 12717 --- [metric-archive-async-16,,] c.o.o.m.service.OcpMetricArchiveService : Archive second nodes failed. org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [REPLACE INTO `metric_data_async` (`record_id`, `timestamp`, `data`, `interval`,`measurement_id`) VALUES (?, ?, ?, ?, ?)]; SQL state [HY000]; error code [4012]; (conn=1933703884) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again.; nested exception is java.sql.BatchUpdateException: (conn=1933703884) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1578) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:669) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:693) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:1036) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:1090) at com.oceanbase.ocp.monitor.storage.AsyncComputeDataDao.write(AsyncComputeDataDao.java:66) at com.oceanbase.ocp.monitor.storage.AsyncComputeDataDao$$FastClassBySpringCGLIB$$87cf8d28.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707) at com.oceanbase.ocp.monitor.storage.AsyncComputeDataDao$$EnhancerBySpringCGLIB$$ed28667e.write() at com.oceanbase.ocp.monitor.service.OcpMetricArchiveService.lambda$archiveAsyncCompute$5(OcpMetricArchiveService.java:231) at com.oceanbase.ocp.common.meter.MeterUtils.time(MeterUtils.java:66) at com.oceanbase.ocp.common.meter.DistributionSummaryGroup.time(DistributionSummaryGroup.java:184) at com.oceanbase.ocp.monitor.service.OcpMetricArchiveService.archiveAsyncCompute(OcpMetricArchiveService.java:222) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.sql.BatchUpdateException: (conn=1933703884) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.OceanBaseStatement.executeBatchExceptionEpilogue(OceanBaseStatement.java:450) at com.oceanbase.jdbc.OceanBaseStatement.executeBatchExceptionEpilogue(OceanBaseStatement.java:453) at com.oceanbase.jdbc.JDBC4PreparedStatement.executeBatch(JDBC4PreparedStatement.java:346) at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:127) at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java) at org.springframework.jdbc.core.JdbcTemplate.lambda$batchUpdate$4(JdbcTemplate.java:1050) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:650) ... 25 common frames omitted Caused by: java.sql.SQLTransientConnectionException: (conn=1933703884) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:128) at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:202) at com.oceanbase.jdbc.OceanBaseStatement.executeBatchExceptionEpilogue(OceanBaseStatement.java:446) ... 31 common frames omitted Caused by: com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException: Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException.of(OceanBaseSqlException.java:79) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeBatchRewrite(AbstractQueryProtocol.java:1252) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeBatchClient(AbstractQueryProtocol.java:704) at com.oceanbase.jdbc.JDBC4PreparedStatement.executeInternalBatch(JDBC4PreparedStatement.java:461) at com.oceanbase.jdbc.JDBC4PreparedStatement.executeBatch(JDBC4PreparedStatement.java:332) ... 29 common frames omitted Caused by: java.sql.SQLException: Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.readErrorPacket(AbstractQueryProtocol.java:2366) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.readPacket(AbstractQueryProtocol.java:2229) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:2119) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeBatchRewrite(AbstractQueryProtocol.java:1244) ... 32 common frames omitted 2025-11-29 08:44:28.258 INFO 12717 --- [ocp-schedules-16,6de1fa233d3c3615,a555ff037e7eb221] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:44:28.258 INFO 12717 --- [ocp-schedules-16,6de1fa233d3c3615,a555ff037e7eb221] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:44:28.259 ERROR 12717 --- [metric-archive-async-13,,] c.o.o.m.service.OcpMetricArchiveService : Archive second nodes failed. org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [REPLACE INTO `metric_data_async` (`record_id`, `timestamp`, `data`, `interval`,`measurement_id`) VALUES (?, ?, ?, ?, ?)]; SQL state [HY000]; error code [4012]; (conn=1933651306) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again.; nested exception is java.sql.BatchUpdateException: (conn=1933651306) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1578) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:669) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:693) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:1036) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:1090) at com.oceanbase.ocp.monitor.storage.AsyncComputeDataDao.write(AsyncComputeDataDao.java:66) at com.oceanbase.ocp.monitor.storage.AsyncComputeDataDao$$FastClassBySpringCGLIB$$87cf8d28.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707) at com.oceanbase.ocp.monitor.storage.AsyncComputeDataDao$$EnhancerBySpringCGLIB$$ed28667e.write() at com.oceanbase.ocp.monitor.service.OcpMetricArchiveService.lambda$archiveAsyncCompute$5(OcpMetricArchiveService.java:231) at com.oceanbase.ocp.common.meter.MeterUtils.time(MeterUtils.java:66) at com.oceanbase.ocp.common.meter.DistributionSummaryGroup.time(DistributionSummaryGroup.java:184) at com.oceanbase.ocp.monitor.service.OcpMetricArchiveService.archiveAsyncCompute(OcpMetricArchiveService.java:222) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.sql.BatchUpdateException: (conn=1933651306) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.OceanBaseStatement.executeBatchExceptionEpilogue(OceanBaseStatement.java:450) at com.oceanbase.jdbc.OceanBaseStatement.executeBatchExceptionEpilogue(OceanBaseStatement.java:453) at com.oceanbase.jdbc.JDBC4PreparedStatement.executeBatch(JDBC4PreparedStatement.java:346) at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:127) at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java) at org.springframework.jdbc.core.JdbcTemplate.lambda$batchUpdate$4(JdbcTemplate.java:1050) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:650) ... 25 common frames omitted Caused by: java.sql.SQLTransientConnectionException: (conn=1933651306) Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:128) at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:202) at com.oceanbase.jdbc.OceanBaseStatement.executeBatchExceptionEpilogue(OceanBaseStatement.java:446) ... 31 common frames omitted Caused by: com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException: Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException.of(OceanBaseSqlException.java:79) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeBatchRewrite(AbstractQueryProtocol.java:1252) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeBatchClient(AbstractQueryProtocol.java:704) at com.oceanbase.jdbc.JDBC4PreparedStatement.executeInternalBatch(JDBC4PreparedStatement.java:461) at com.oceanbase.jdbc.JDBC4PreparedStatement.executeBatch(JDBC4PreparedStatement.java:332) ... 29 common frames omitted Caused by: java.sql.SQLException: Timeout, query has reached the maximum query timeout: 10000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again. at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.readErrorPacket(AbstractQueryProtocol.java:2366) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.readPacket(AbstractQueryProtocol.java:2229) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:2119) at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeBatchRewrite(AbstractQueryProtocol.java:1244) ... 32 common frames omitted 2025-11-29 08:44:30.145 INFO 12717 --- [ocp-async-22,c8309fdc460a2b45,c3af456de31b866b] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:30.145 INFO 12717 --- [prometheus-minute-schedule-1,a3b532902fc513f2,a68faa1c62748a2a] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:30.147 INFO 12717 --- [prometheus-second-schedule-1,d448206794111646,f78c153a6dd120a5] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=42, collectTaskCount=42 2025-11-29 08:44:30.166 INFO 12717 --- [ocp-schedules-26,cb8e246af1489130,e2640148ffdfbf22] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:44:30.177 INFO 12717 --- [prometheus-minute-schedule-1,a3b532902fc513f2,a68faa1c62748a2a] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:30.198 INFO 12717 --- [ocp-schedules-26,cb8e246af1489130,e2640148ffdfbf22] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:44:30.249 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:44:30.247713+08:00, lastProcessTime=2025-11-29T08:44:20.161395+08:00 2025-11-29 08:44:30.271 INFO 12717 --- [ocp-schedules-22,88a576d9a0783efa,040e104099fb0d42] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-11-29 08:44:30.271 INFO 12717 --- [ocp-schedules-25,bca80dc8a1ffbe3a,e595cc4b6dc51062] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-11-29 08:44:30.271 INFO 12717 --- [ocp-schedules-8,dfb3446ff676c407,22ece1e257f792a8] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-11-29 08:44:30.273 INFO 12717 --- [ocp-schedules-8,dfb3446ff676c407,22ece1e257f792a8] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:44:30.273 INFO 12717 --- [ocp-schedules-25,bca80dc8a1ffbe3a,e595cc4b6dc51062] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:44:30.291 INFO 12717 --- [ocp-schedules-9,453c5e233e058c11,63ecbdcdeadf3290] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:44:30.306 INFO 12717 --- [ocp-schedules-15,5b0965406c0c2698,3ada6ecd29a40314] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:44:30.308 INFO 12717 --- [host-check-agent-status1,5b0965406c0c2698,c7050561761c9448] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:44:30.308 INFO 12717 --- [host-check-agent-status4,5b0965406c0c2698,89047424dd6010c2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:44:30.308 INFO 12717 --- [host-check-agent-status2,5b0965406c0c2698,f64511751332b50b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:44:30.308 INFO 12717 --- [host-check-agent-status5,5b0965406c0c2698,5a5e71220fb44a41] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:44:30.309 INFO 12717 --- [host-check-agent-status7,5b0965406c0c2698,5bd46a412f336e92] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:44:30.310 INFO 12717 --- [host-check-agent-status8,5b0965406c0c2698,629736a9408e6495] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:44:30.310 INFO 12717 --- [host-check-agent-status9,5b0965406c0c2698,f8acf58e5630b301] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:44:30.310 INFO 12717 --- [host-check-agent-status10,5b0965406c0c2698,ecda72952b397a2f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:44:30.310 INFO 12717 --- [host-check-agent-status11,5b0965406c0c2698,936781b025559894] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:44:30.312 INFO 12717 --- [host-check-agent-status6,5b0965406c0c2698,08b9da331525a6f3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:44:30.313 INFO 12717 --- [host-check-agent-status3,5b0965406c0c2698,0fc47ea45cd43683] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:44:30.313 INFO 12717 --- [host-check-agent-status12,5b0965406c0c2698,91e0bfee93e97980] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:44:30.354 INFO 12717 --- [ocp-schedules-30,ac22a60957b1e912,94d7993c88e317a0] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:44:30.355 INFO 12717 --- [ocp-schedules-7,928390af0d622862,c90154dd80767926] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:44:30.357 INFO 12717 --- [host-check-agent-status8,5b0965406c0c2698,629736a9408e6495] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:44:30.357 INFO 12717 --- [host-check-agent-status6,5b0965406c0c2698,08b9da331525a6f3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:44:30.357 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:44:30.357 INFO 12717 --- [host-check-agent-status2,5b0965406c0c2698,f64511751332b50b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:44:30.358 INFO 12717 --- [host-check-clockdiff5,928390af0d622862,bb5b52fb9eca8843] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.53], timeoutMillis=20000 2025-11-29 08:44:30.359 INFO 12717 --- [host-check-agent-status10,5b0965406c0c2698,ecda72952b397a2f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:44:30.360 INFO 12717 --- [host-check-agent-status5,5b0965406c0c2698,5a5e71220fb44a41] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:44:30.360 INFO 12717 --- [host-check-clockdiff8,928390af0d622862,69394cd8d20fa6e5] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.60], timeoutMillis=20000 2025-11-29 08:44:30.360 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:44:30.360 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:44:30.360 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:44:30.360 INFO 12717 --- [host-check-clockdiff7,928390af0d622862,12b8c6dc687f5ea2] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.59], timeoutMillis=20000 2025-11-29 08:44:30.360 INFO 12717 --- [host-check-clockdiff1,928390af0d622862,eead10d0f56c179f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.58], timeoutMillis=20000 2025-11-29 08:44:30.360 INFO 12717 --- [host-check-clockdiff6,928390af0d622862,5584f6fb4256fa64] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.54], timeoutMillis=20000 2025-11-29 08:44:30.361 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:44:30.364 INFO 12717 --- [host-check-agent-status12,5b0965406c0c2698,91e0bfee93e97980] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:44:30.364 INFO 12717 --- [host-check-agent-status3,5b0965406c0c2698,0fc47ea45cd43683] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:44:30.365 INFO 12717 --- [host-check-agent-status6,5b0965406c0c2698,08b9da331525a6f3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:44:30.365 INFO 12717 --- [host-check-agent-status2,5b0965406c0c2698,f64511751332b50b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:44:30.366 INFO 12717 --- [host-check-agent-status8,5b0965406c0c2698,629736a9408e6495] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:44:30.366 INFO 12717 --- [host-check-agent-status3,5b0965406c0c2698,0fc47ea45cd43683] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:44:30.366 INFO 12717 --- [host-check-agent-status4,5b0965406c0c2698,89047424dd6010c2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:44:30.366 INFO 12717 --- [host-check-clockdiff4,928390af0d622862,a7b9c01ddc692ad3] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.52], timeoutMillis=20000 2025-11-29 08:44:30.366 INFO 12717 --- [host-check-agent-status5,5b0965406c0c2698,5a5e71220fb44a41] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:44:30.367 INFO 12717 --- [host-check-agent-status10,5b0965406c0c2698,ecda72952b397a2f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:44:30.367 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:44:30.368 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:44:30.368 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:44:30.368 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:44:30.368 INFO 12717 --- [host-check-agent-status12,5b0965406c0c2698,91e0bfee93e97980] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:44:30.369 INFO 12717 --- [host-check-agent-status11,5b0965406c0c2698,936781b025559894] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:44:30.369 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:44:30.369 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:44:30.371 INFO 12717 --- [host-check-clockdiff3,928390af0d622862,607e0c693c2ef1b4] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.51], timeoutMillis=20000 2025-11-29 08:44:30.371 INFO 12717 --- [host-check-clockdiff2,928390af0d622862,0583236f193e0ee3] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.50], timeoutMillis=20000 2025-11-29 08:44:30.371 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:44:30.371 INFO 12717 --- [host-check-clockdiff10,928390af0d622862,cbbdddb5a2636627] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.47], timeoutMillis=20000 2025-11-29 08:44:30.371 INFO 12717 --- [host-check-clockdiff9,928390af0d622862,cb4ab887246535a6] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.61], timeoutMillis=20000 2025-11-29 08:44:30.372 INFO 12717 --- [host-check-agent-status4,5b0965406c0c2698,89047424dd6010c2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:44:30.372 INFO 12717 --- [host-check-agent-status1,5b0965406c0c2698,c7050561761c9448] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:44:30.373 INFO 12717 --- [host-check-clockdiff11,928390af0d622862,38479f22a84d68b6] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.48], timeoutMillis=20000 2025-11-29 08:44:30.373 INFO 12717 --- [host-check-agent-status11,5b0965406c0c2698,936781b025559894] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:44:30.374 INFO 12717 --- [host-check-agent-status1,5b0965406c0c2698,c7050561761c9448] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:44:30.376 INFO 12717 --- [host-check-clockdiff12,928390af0d622862,1e0882400638053e] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.49], timeoutMillis=20000 2025-11-29 08:44:30.382 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:44:30.383 INFO 12717 --- [host-check-agent-status9,5b0965406c0c2698,f8acf58e5630b301] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:44:30.409 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,8812befd7df096ff,bb3de5d7552900c1] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=8812befd7df096ff, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:30.423 INFO 12717 --- [host-check-clockdiff1,928390af0d622862,eead10d0f56c179f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764377070 0 0 , error= 2025-11-29 08:44:30.410 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,8812befd7df096ff,bb3de5d7552900c1] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=8812befd7df096ff, duration=2 ms] 2025-11-29 08:44:30.470 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:44:30.471 INFO 12717 --- [host-check-agent-status7,5b0965406c0c2698,5bd46a412f336e92] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:44:30.473 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:44:30.473 INFO 12717 --- [host-check-agent-status9,5b0965406c0c2698,f8acf58e5630b301] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:44:30.476 INFO 12717 --- [ocp-schedules-17,39e9a24eaf080dc2,012c049e0cd88a30] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-11-29 08:44:30.481 INFO 12717 --- [host-check-agent-status7,5b0965406c0c2698,5bd46a412f336e92] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:44:30.482 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:44:30.483 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:44:30.483 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:44:30.483 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:44:30.483 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:44:30.483 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:44:30.484 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:44:30.484 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:44:30.485 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:44:30.486 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:44:30.487 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:44:30.487 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:44:30.487 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:44:30.487 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:44:30.488 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:44:30.488 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:44:30.490 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:44:30.490 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:44:30.504 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:44:30.504 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:44:30.506 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:44:30.517 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:30.517 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:44:30.535 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:30.537 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:44:30.538 INFO 12717 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.8 2025-11-29 08:44:30.552 INFO 12717 --- [host-check-agent-status10,5b0965406c0c2698,ecda72952b397a2f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.553 INFO 12717 --- [host-check-agent-status8,5b0965406c0c2698,629736a9408e6495] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.554 INFO 12717 --- [host-check-agent-status4,5b0965406c0c2698,89047424dd6010c2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.554 INFO 12717 --- [host-check-agent-status6,5b0965406c0c2698,08b9da331525a6f3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.554 INFO 12717 --- [host-check-agent-status3,5b0965406c0c2698,0fc47ea45cd43683] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.554 INFO 12717 --- [host-check-agent-status1,5b0965406c0c2698,c7050561761c9448] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.556 INFO 12717 --- [host-check-agent-status2,5b0965406c0c2698,f64511751332b50b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.556 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/time, params:null 2025-11-29 08:44:30.558 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/time, params:null 2025-11-29 08:44:30.558 INFO 12717 --- [host-check-agent-status3,5b0965406c0c2698,0fc47ea45cd43683] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.51 2025-11-29 08:44:30.558 INFO 12717 --- [host-check-agent-status3,5b0965406c0c2698,0fc47ea45cd43683] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.51 2025-11-29 08:44:30.558 INFO 12717 --- [host-check-agent-status8,5b0965406c0c2698,629736a9408e6495] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.60 2025-11-29 08:44:30.558 INFO 12717 --- [host-check-agent-status3,5b0965406c0c2698,0fc47ea45cd43683] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.51 2025-11-29 08:44:30.558 INFO 12717 --- [host-check-agent-status8,5b0965406c0c2698,629736a9408e6495] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.60 2025-11-29 08:44:30.559 INFO 12717 --- [host-check-agent-status8,5b0965406c0c2698,629736a9408e6495] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.60 2025-11-29 08:44:30.559 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/time, params:null 2025-11-29 08:44:30.559 INFO 12717 --- [host-check-agent-status5,5b0965406c0c2698,5a5e71220fb44a41] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.559 INFO 12717 --- [host-check-agent-status10,5b0965406c0c2698,ecda72952b397a2f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.47 2025-11-29 08:44:30.559 INFO 12717 --- [host-check-agent-status10,5b0965406c0c2698,ecda72952b397a2f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.47 2025-11-29 08:44:30.559 INFO 12717 --- [host-check-agent-status10,5b0965406c0c2698,ecda72952b397a2f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.47 2025-11-29 08:44:30.559 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=9, ip=10.202.22.60, timestamp=2025-11-29T08:44:30.553702915+08:00 2025-11-29 08:44:30.559 INFO 12717 --- [host-check-agent-status7,5b0965406c0c2698,5bd46a412f336e92] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.559 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.202.22.50, timestamp=2025-11-29T08:44:30.559823934+08:00 2025-11-29 08:44:30.560 INFO 12717 --- [host-check-agent-status6,5b0965406c0c2698,08b9da331525a6f3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.54 2025-11-29 08:44:30.560 INFO 12717 --- [host-check-agent-status6,5b0965406c0c2698,08b9da331525a6f3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.54 2025-11-29 08:44:30.560 INFO 12717 --- [host-check-agent-status6,5b0965406c0c2698,08b9da331525a6f3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.54 2025-11-29 08:44:30.560 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.202.22.51, timestamp=2025-11-29T08:44:30.557208833+08:00 2025-11-29 08:44:30.560 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/time, params:null 2025-11-29 08:44:30.560 INFO 12717 --- [host-check-agent-status4,5b0965406c0c2698,89047424dd6010c2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.52 2025-11-29 08:44:30.560 INFO 12717 --- [host-check-agent-status4,5b0965406c0c2698,89047424dd6010c2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.52 2025-11-29 08:44:30.560 INFO 12717 --- [host-check-agent-status4,5b0965406c0c2698,89047424dd6010c2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.52 2025-11-29 08:44:30.560 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-11-29T08:44:30.560+08:00 2025-11-29 08:44:30.561 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/time, params:null 2025-11-29 08:44:30.561 INFO 12717 --- [host-check-agent-status2,5b0965406c0c2698,f64511751332b50b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.50 2025-11-29 08:44:30.561 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/time, params:null 2025-11-29 08:44:30.561 INFO 12717 --- [host-check-agent-status2,5b0965406c0c2698,f64511751332b50b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.50 2025-11-29 08:44:30.561 INFO 12717 --- [host-check-agent-status2,5b0965406c0c2698,f64511751332b50b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.50 2025-11-29 08:44:30.561 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=11, ip=10.202.22.47, timestamp=2025-11-29T08:44:30.560805432+08:00 2025-11-29 08:44:30.562 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/time, params:null 2025-11-29 08:44:30.562 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=11, lastAvailableTime=2025-11-29T08:44:30.561+08:00 2025-11-29 08:44:30.562 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.202.22.58, timestamp=2025-11-29T08:44:30.562440660+08:00 2025-11-29 08:44:30.563 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-11-29T08:44:30.560+08:00 2025-11-29 08:44:30.563 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=9, lastAvailableTime=2025-11-29T08:44:30.559+08:00 2025-11-29 08:44:30.563 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-11-29T08:44:30.562+08:00 2025-11-29 08:44:30.563 INFO 12717 --- [host-check-agent-status11,5b0965406c0c2698,936781b025559894] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.563 INFO 12717 --- [host-check-agent-status1,5b0965406c0c2698,c7050561761c9448] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.58 2025-11-29 08:44:30.563 INFO 12717 --- [host-check-agent-status1,5b0965406c0c2698,c7050561761c9448] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.58 2025-11-29 08:44:30.563 INFO 12717 --- [host-check-agent-status1,5b0965406c0c2698,c7050561761c9448] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.58 2025-11-29 08:44:30.564 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2025-11-29T08:44:30.560+08:00 and status AVAILABLE 2025-11-29 08:44:30.564 INFO 12717 --- [host-check-agent-status9,5b0965406c0c2698,f8acf58e5630b301] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.565 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/time, params:null 2025-11-29 08:44:30.565 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/time, params:null 2025-11-29 08:44:30.565 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/time, params:null 2025-11-29 08:44:30.565 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.s.compute.HostAgentDaoManager : Updated agent 11 last available time 2025-11-29T08:44:30.561+08:00 and status AVAILABLE 2025-11-29 08:44:30.565 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/time, params:null 2025-11-29 08:44:30.566 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.202.22.53, timestamp=2025-11-29T08:44:30.564188056+08:00 2025-11-29 08:44:30.566 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.202.22.54, timestamp=2025-11-29T08:44:30.564453458+08:00 2025-11-29 08:44:30.566 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.202.22.52, timestamp=2025-11-29T08:44:30.564093822+08:00 2025-11-29 08:44:30.566 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/time, params:null 2025-11-29 08:44:30.567 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-11-29T08:44:30.566+08:00 2025-11-29 08:44:30.567 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=13, ip=10.202.22.49, timestamp=2025-11-29T08:44:30.565510309+08:00 2025-11-29 08:44:30.567 INFO 12717 --- [host-check-agent-status5,5b0965406c0c2698,5a5e71220fb44a41] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.53 2025-11-29 08:44:30.567 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-11-29T08:44:30.566+08:00 2025-11-29 08:44:30.567 INFO 12717 --- [host-check-agent-status5,5b0965406c0c2698,5a5e71220fb44a41] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.53 2025-11-29 08:44:30.568 INFO 12717 --- [host-check-agent-status5,5b0965406c0c2698,5a5e71220fb44a41] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.53 2025-11-29 08:44:30.568 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-11-29T08:44:30.560+08:00 and status AVAILABLE 2025-11-29 08:44:30.568 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.s.compute.HostAgentDaoManager : Updated agent 10 last available time 2025-11-29T08:44:30.559+08:00 and status AVAILABLE 2025-11-29 08:44:30.568 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2025-11-29T08:44:30.566+08:00 2025-11-29 08:44:30.568 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=10, ip=10.202.22.61, timestamp=2025-11-29T08:44:30.565931056+08:00 2025-11-29 08:44:30.568 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=13, lastAvailableTime=2025-11-29T08:44:30.567+08:00 2025-11-29 08:44:30.569 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-11-29T08:44:30.562+08:00 and status AVAILABLE 2025-11-29 08:44:30.569 INFO 12717 --- [host-check-agent-status9,5b0965406c0c2698,f8acf58e5630b301] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.61 2025-11-29 08:44:30.569 INFO 12717 --- [host-check-agent-status9,5b0965406c0c2698,f8acf58e5630b301] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.61 2025-11-29 08:44:30.569 INFO 12717 --- [host-check-agent-status9,5b0965406c0c2698,f8acf58e5630b301] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.61 2025-11-29 08:44:30.570 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=10, lastAvailableTime=2025-11-29T08:44:30.568+08:00 2025-11-29 08:44:30.570 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-11-29T08:44:30.566+08:00 and status AVAILABLE 2025-11-29 08:44:30.572 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.s.compute.HostAgentDaoManager : Updated agent 12 last available time 2025-11-29T08:44:30.567+08:00 and status AVAILABLE 2025-11-29 08:44:30.572 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-11-29T08:44:30.566+08:00 and status AVAILABLE 2025-11-29 08:44:30.572 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-11-29T08:44:30.566+08:00 and status AVAILABLE 2025-11-29 08:44:30.574 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-11-29 08:44:30.576 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2025-11-29T08:44:30.568+08:00 and status AVAILABLE 2025-11-29 08:44:30.576 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-11-29 08:44:30.576 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-11-29 08:44:30.576 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=11 2025-11-29 08:44:30.577 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-11-29 08:44:30.578 INFO 12717 --- [host-check-agent-status12,5b0965406c0c2698,91e0bfee93e97980] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/agent/status, params:null 2025-11-29 08:44:30.578 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=13 2025-11-29 08:44:30.578 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-11-29 08:44:30.578 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2025-11-29 08:44:30.578 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=9 2025-11-29 08:44:30.581 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=12, ip=10.202.22.48, timestamp=2025-11-29T08:44:30.578577961+08:00 2025-11-29 08:44:30.582 INFO 12717 --- [host-check-agent-status12,5b0965406c0c2698,91e0bfee93e97980] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.49 2025-11-29 08:44:30.582 INFO 12717 --- [host-check-agent-status12,5b0965406c0c2698,91e0bfee93e97980] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.49 2025-11-29 08:44:30.582 INFO 12717 --- [host-check-agent-status12,5b0965406c0c2698,91e0bfee93e97980] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.49 2025-11-29 08:44:30.585 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=10 2025-11-29 08:44:30.585 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=12, lastAvailableTime=2025-11-29T08:44:30.581+08:00 2025-11-29 08:44:30.586 INFO 12717 --- [host-check-agent-status11,5b0965406c0c2698,936781b025559894] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.48 2025-11-29 08:44:30.586 INFO 12717 --- [host-check-agent-status11,5b0965406c0c2698,936781b025559894] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.48 2025-11-29 08:44:30.586 INFO 12717 --- [host-check-agent-status11,5b0965406c0c2698,936781b025559894] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.48 2025-11-29 08:44:30.609 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.s.compute.HostAgentDaoManager : Updated agent 13 last available time 2025-11-29T08:44:30.581+08:00 and status AVAILABLE 2025-11-29 08:44:30.635 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:44:30.635 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:44:30.635 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:44:30.635 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:44:30.636 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:44:30.637 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:44:30.637 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:44:30.637 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:44:30.637 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:44:30.637 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:44:30.638 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:44:30.638 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:44:30.638 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:44:30.639 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:44:30.640 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:44:30.640 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:44:30.641 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:44:30.641 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:44:30.645 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:44:30.649 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:44:30.670 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=8, ip=10.202.22.59, timestamp=2025-11-29T08:44:30.669686977+08:00 2025-11-29 08:44:30.672 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=8, lastAvailableTime=2025-11-29T08:44:30.671+08:00 2025-11-29 08:44:30.673 INFO 12717 --- [Check status for all clusters2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.3.5.1 2025-11-29 08:44:30.674 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:30.674 INFO 12717 --- [host-check-agent-status7,5b0965406c0c2698,5bd46a412f336e92] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.59 2025-11-29 08:44:30.674 INFO 12717 --- [host-check-agent-status7,5b0965406c0c2698,5bd46a412f336e92] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.59 2025-11-29 08:44:30.674 INFO 12717 --- [host-check-agent-status7,5b0965406c0c2698,5bd46a412f336e92] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.59 2025-11-29 08:44:30.675 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.253 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:32.257 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.s.compute.HostAgentDaoManager : Updated agent 8 last available time 2025-11-29T08:44:30.671+08:00 and status AVAILABLE 2025-11-29 08:44:32.259 INFO 12717 --- [ocp-async-28,a649635bf85fa3e8,14265796b200e096] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:32.260 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=16 2025-11-29 08:44:32.261 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=28 2025-11-29 08:44:32.265 INFO 12717 --- [ocp-schedules-12,9ae9389fc1278a64,8a5b29c0ee076773] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:44:32.265 INFO 12717 --- [ocp-schedules-12,06d47c0ecf950907,6cd5be2769105c81] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-11-29 08:44:32.267 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.269 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.272 INFO 12717 --- [ocp-schedules-12,06d47c0ecf950907,6cd5be2769105c81] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:44:32.273 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:32.274 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.276 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.282 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.289 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.290 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.291 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.322 WARN 12717 --- [host-check-clockdiff12,928390af0d622862,1e0882400638053e] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.49], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:32.322 ERROR 12717 --- [host-check-clockdiff12,928390af0d622862,1e0882400638053e] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.49] 2025-11-29 08:44:32.325 WARN 12717 --- [host-check-clockdiff12,928390af0d622862,1e0882400638053e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=13, ip=10.202.22.49, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.49 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:32.325 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-11-29 08:44:32.325 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-11-29 08:44:32.330 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-11-29 08:44:32.337 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=34 2025-11-29 08:44:32.348 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=31 2025-11-29 08:44:32.366 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-11-29 08:44:32.368 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-11-29 08:44:32.397 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=28 2025-11-29 08:44:32.397 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=16 2025-11-29 08:44:32.407 INFO 12717 --- [prometheus-minute-schedule-1,74bf9ea2e8c1f932,bfa64e554669f6ab] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:32.410 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-11-29 08:44:32.416 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-11-29 08:44:32.416 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-11-29 08:44:32.417 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=34 2025-11-29 08:44:32.418 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-11-29 08:44:32.418 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=31 2025-11-29 08:44:32.420 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-11-29 08:44:32.425 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:32.425 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:44:32.426 INFO 12717 --- [prometheus-minute-schedule-1,74bf9ea2e8c1f932,bfa64e554669f6ab] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:32.449 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:32.450 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:44:32.461 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=12 2025-11-29 08:44:32.467 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=8 2025-11-29 08:44:32.476 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,17bebeb54b862aae,11bc040dc96df62a] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=17bebeb54b862aae, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:32.486 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,17bebeb54b862aae,11bc040dc96df62a] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=17bebeb54b862aae, duration=11 ms] 2025-11-29 08:44:32.493 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:44:32.493 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:44:32.497 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.499 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.500 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.501 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.501 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:32.501 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.501 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.502 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:44:32.502 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.502 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:44:32.502 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:32.502 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.503 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:44:32.507 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.530 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=25 2025-11-29 08:44:32.542 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.543 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:44:32.548 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=17 2025-11-29 08:44:32.548 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=25 2025-11-29 08:44:32.551 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-11-29 08:44:32.562 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=17 2025-11-29 08:44:32.562 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=35 2025-11-29 08:44:32.565 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=29 2025-11-29 08:44:32.566 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-11-29 08:44:32.568 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-11-29 08:44:32.568 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=32 2025-11-29 08:44:32.568 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-11-29 08:44:32.570 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-11-29 08:44:32.570 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:32.572 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:32.574 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:44:32.576 WARN 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=OCP 2025-11-29 08:44:32.576 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=35 2025-11-29 08:44:32.577 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:32.580 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:44:32.582 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:44:32.583 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-11-29 08:44:32.583 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=29 2025-11-29 08:44:32.586 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-11-29 08:44:32.586 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-11-29 08:44:32.589 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-11-29 08:44:32.590 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.590 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=22 2025-11-29 08:44:32.591 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=32 2025-11-29 08:44:32.600 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:32.600 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=OCP, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:32.601 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-11-29 08:44:32.601 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:32.607 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=22 2025-11-29 08:44:32.610 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:32.612 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:32.612 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:32.614 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:32.614 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:32.617 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:32.621 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:32.626 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.202.22.58, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.2.1.8_108000022024072217-3149c25ca2dadbb7707686ad02a1367b1b43e0b5(Jul 23 2024 03:19:19), withRootserver=true, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T10:35:04.127801+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:32.628 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:32.633 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:32.639 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:32.640 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=26 2025-11-29 08:44:32.640 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:32.641 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:44:32.642 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster ocpserver 2025-11-29 08:44:32.648 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2025-11-29 08:44:32.651 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=18 2025-11-29 08:44:32.653 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=26 2025-11-29 08:44:32.656 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:44:32.659 INFO 12717 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:44:32.661 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-11-29 08:44:32.664 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-11-29 08:44:32.665 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=37 2025-11-29 08:44:32.667 INFO 12717 --- [host-check-host-status2,ac22a60957b1e912,e6d07c55db7e6f16] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=18 2025-11-29 08:44:32.669 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=30 2025-11-29 08:44:32.669 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=36 2025-11-29 08:44:32.670 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-11-29 08:44:32.672 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-11-29 08:44:32.680 INFO 12717 --- [host-check-host-status6,ac22a60957b1e912,93c27911c112d921] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-11-29 08:44:32.680 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=37 2025-11-29 08:44:32.680 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:32.684 INFO 12717 --- [host-check-host-status5,ac22a60957b1e912,08aa6a53517b2158] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-11-29 08:44:32.685 INFO 12717 --- [host-check-host-status3,ac22a60957b1e912,5f3c22801e971e1b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-11-29 08:44:32.686 INFO 12717 --- [host-check-host-status12,ac22a60957b1e912,21667edd0b7d4728] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=36 2025-11-29 08:44:32.687 INFO 12717 --- [host-check-host-status4,ac22a60957b1e912,982c6881735e1b29] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-11-29 08:44:32.689 INFO 12717 --- [host-check-host-status8,ac22a60957b1e912,5a54bf4a37b44d87] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=30 2025-11-29 08:44:32.697 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=33 2025-11-29 08:44:32.699 INFO 12717 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:44:32.710 INFO 12717 --- [host-check-host-status10,ac22a60957b1e912,dc7c6964310c5f17] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=33 2025-11-29 08:44:32.725 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-11-29 08:44:32.739 INFO 12717 --- [host-check-host-status1,ac22a60957b1e912,a247d9d73958b4e7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-11-29 08:44:32.741 WARN 12717 --- [host-check-clockdiff4,928390af0d622862,a7b9c01ddc692ad3] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.52], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:32.741 ERROR 12717 --- [host-check-clockdiff4,928390af0d622862,a7b9c01ddc692ad3] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.52] 2025-11-29 08:44:32.741 WARN 12717 --- [host-check-clockdiff4,928390af0d622862,a7b9c01ddc692ad3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.202.22.52, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.52 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:34.274 INFO 12717 --- [ocp-async-37,a5842d8a94754e22,900a7e63f63a176a] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:44:34.276 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=23 2025-11-29 08:44:34.278 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,64859add7e569082,275e0e963fb20d9a] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=64859add7e569082, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:34.278 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,955761010ce70e43,f200b858dd0c418f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=955761010ce70e43, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:34.279 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,64859add7e569082,275e0e963fb20d9a] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=64859add7e569082, duration=2 ms] 2025-11-29 08:44:34.279 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=27 2025-11-29 08:44:34.279 INFO 12717 --- [ocp-async-23,8df7cf079b7ba5b5,d828431602814edf] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:34.279 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,955761010ce70e43,f200b858dd0c418f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=955761010ce70e43, duration=2 ms] 2025-11-29 08:44:34.306 INFO 12717 --- [ocp-schedules-26,41d70f941c2e8f48,a3cd53981b6fa58e] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:44:34.307 INFO 12717 --- [ocp-schedules-26,41d70f941c2e8f48,a3cd53981b6fa58e] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=9 2025-11-29 08:44:34.307 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2025-11-29 08:44:34.307 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=6 2025-11-29 08:44:34.307 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2025-11-29 08:44:34.307 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2025-11-29 08:44:34.307 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2025-11-29 08:44:34.307 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=12 2025-11-29 08:44:34.307 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=8 2025-11-29 08:44:34.307 INFO 12717 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2025-11-29 08:44:34.307 INFO 12717 --- [ForkJoinPool.commonPool-worker-12,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=13 2025-11-29 08:44:34.307 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=11 2025-11-29 08:44:34.308 INFO 12717 --- [ForkJoinPool.commonPool-worker-19,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=10 2025-11-29 08:44:34.318 INFO 12717 --- [host-check-host-status9,ac22a60957b1e912,6222d0bc5cfd671f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=27 2025-11-29 08:44:34.320 INFO 12717 --- [ocp-schedules-26,41d70f941c2e8f48,a3cd53981b6fa58e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:44:34.320 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:44:34.321 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:44:34.321 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:44:34.321 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:44:34.321 INFO 12717 --- [ForkJoinPool.commonPool-worker-12,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:44:34.321 INFO 12717 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:44:34.322 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:44:34.323 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:44:34.323 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:44:34.323 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=23 2025-11-29 08:44:34.323 INFO 12717 --- [ocp-schedules-26,41d70f941c2e8f48,a3cd53981b6fa58e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:44:34.323 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:44:34.323 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:44:34.325 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:44:34.325 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:44:34.325 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:44:34.325 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:44:34.326 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:44:34.326 INFO 12717 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:44:34.326 INFO 12717 --- [ForkJoinPool.commonPool-worker-12,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:44:34.326 INFO 12717 --- [ForkJoinPool.commonPool-worker-19,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:44:34.328 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:44:34.328 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:44:34.329 INFO 12717 --- [ForkJoinPool.commonPool-worker-19,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:44:34.357 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:44:30.247713+08:00 2025-11-29 08:44:34.390 INFO 12717 --- [ForkJoinPool.commonPool-worker-27,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.390 INFO 12717 --- [ForkJoinPool.commonPool-worker-25,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.391 INFO 12717 --- [ForkJoinPool.commonPool-worker-20,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.391 INFO 12717 --- [ForkJoinPool.commonPool-worker-10,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.391 INFO 12717 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=8073, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:44:34.393 INFO 12717 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.393 INFO 12717 --- [ForkJoinPool.commonPool-worker-14,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.393 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:44:34.394 INFO 12717 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.397 INFO 12717 --- [ForkJoinPool.commonPool-worker-19,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.398 INFO 12717 --- [ForkJoinPool.commonPool-worker-28,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.398 INFO 12717 --- [ForkJoinPool.commonPool-worker-12,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.399 INFO 12717 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.400 INFO 12717 --- [ocp-schedules-26,41d70f941c2e8f48,a3cd53981b6fa58e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62889/api/v1/log/alarms, params:null 2025-11-29 08:44:34.402 INFO 12717 --- [ocp-schedules-26,41d70f941c2e8f48,a3cd53981b6fa58e] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms finished. 2025-11-29 08:44:34.403 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:34.406 INFO 12717 --- [ocp-schedules-1,04cd9084bea831a3,d71ba6d0b26695c3] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:44:34.410 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:34.411 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:34.412 INFO 12717 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:44:34.425 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:34.426 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:34.427 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-11-29 08:44:34.432 WARN 12717 --- [scanner-11213818094-1,04cd9084bea831a3,e30477256cc9bcb4] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:44:34.440 INFO 12717 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=ocpserver, obClusterId=1743215593 2025-11-29 08:44:34.440 INFO 12717 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=ocpserver, obClusterId=1743215593, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"ocpserver","ObClusterId":1743215593,"ObRegion":"ocpserver","ObRegionId":1743215593,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.58:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376673675000}, createTime=2025-03-29T10:43:06+08:00, creator=admin, obVersion=4.2.1.8, packageOperatingSystem=null, loadType=null, tenantCount=3) 2025-11-29 08:44:34.472 INFO 12717 --- [alarm-detect-18,04cd9084bea831a3,ab02db6c464c6e06] c.o.o.alarm.core.detect.MetricDetector : alarmType ob_tenant_cpu_usage_over_threshold detect time range 1764377040 ~ 1764377064 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricSnapshots={ob_tenant_cpu_percent=[SeriesMetricValues(metric=Metric{name='ob_tenant_cpu_percent', labels={app=OB, host=10.202.22.58, host_name=ocp-22.58, ob_cluster_id=1743215593, ob_cluster_name=ocpserver, ob_tenant_id=1004, obzone=OCP, svr_ip=10.202.22.58, tenant_name=ocp_monitor}}, data=[ValueData(timestamp=1764377055, value=178.0)])]}, metricEvaluateResults={ob_tenant_cpu_percent > 95=[ExpressionEvaluateResult(matchValue=95.0, metric=ob_tenant_cpu_percent, match=true, value=178.0, values={1764377055=178.0}, valueCount=1, labels={app=OB, host=10.202.22.58, host_name=ocp-22.58, ob_cluster_id=1743215593, ob_cluster_name=ocpserver, ob_tenant_id=1004, obzone=OCP, svr_ip=10.202.22.58, tenant_name=ocp_monitor})]}, matched=true, isContainValidValue=null, message=null, level=2) 2025-11-29 08:44:34.473 INFO 12717 --- [alarm-detect-18,04cd9084bea831a3,ab02db6c464c6e06] c.o.o.a.c.detect.AlarmRuleDetectState : firstDetect, alarmType=ob_tenant_cpu_usage_over_threshold, target=alarm_template_id=1:ob_cluster=ocpserver-1743215593:tenant_name=ocp_monitor:host=10.202.22.58-2, currentState=Pending 2025-11-29 08:44:34.530 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=38 2025-11-29 08:44:34.542 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=38 2025-11-29 08:44:34.545 WARN 12717 --- [host-check-clockdiff6,928390af0d622862,5584f6fb4256fa64] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.54], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:34.545 ERROR 12717 --- [host-check-clockdiff6,928390af0d622862,5584f6fb4256fa64] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.54] 2025-11-29 08:44:34.545 WARN 12717 --- [host-check-clockdiff6,928390af0d622862,5584f6fb4256fa64] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.202.22.54, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.54 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:34.662 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=24 2025-11-29 08:44:34.669 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:34.670 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:34.671 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-11-29 08:44:34.677 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:34.679 INFO 12717 --- [host-check-host-status7,ac22a60957b1e912,2489fc9e29b8e1ab] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=24 2025-11-29 08:44:34.683 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-11-29 08:44:34.691 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:44:34.713 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone_1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:34.713 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:34.725 INFO 12717 --- [ocp-schedules-6,a28884f1afd4827d,7e1e2377d0201ba2] c.o.o.s.c.scheduled.TenantSchedules : Sync all tenant info schedule. 2025-11-29 08:44:34.727 INFO 12717 --- [ocp-schedules-6,a28884f1afd4827d,7e1e2377d0201ba2] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:44:34.729 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:44:34.744 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=8, ip=10.202.22.59, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=8, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T21:00:26.604920+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:34.744 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:12+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:36.347 INFO 12717 --- [prometheus-minute-schedule-1,f3c849cb38edf15c,a3ea554cb26cba16] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:34.771 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=10, ip=10.202.22.60, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=9, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T15:25:11.554665+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:36.351 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:36.362 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=39 2025-11-29 08:44:36.365 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:36.373 INFO 12717 --- [ocp-async-33,6042df153baba81a,406b2562e2298b55] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:36.392 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:36.405 INFO 12717 --- [host-check-host-status11,ac22a60957b1e912,4716a02099610958] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=39 2025-11-29 08:44:36.407 INFO 12717 --- [prometheus-minute-schedule-1,f3c849cb38edf15c,a3ea554cb26cba16] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:36.426 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-11-29 08:44:36.426 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.432 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-11-29 08:44:36.445 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-11-29 08:44:36.446 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.448 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.449 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-11-29 08:44:36.449 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-11-29 08:44:36.469 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.470 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-11-29 08:44:36.491 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.492 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2025-11-29 08:44:36.496 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.497 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-11-29 08:44:36.517 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.518 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2025-11-29 08:44:36.538 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.540 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1004] 2025-11-29 08:44:36.545 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=9, ip=10.202.22.61, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=10, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T11:47:27.460011+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:36.545 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:44:36.568 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.568 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1004] 2025-11-29 08:44:36.571 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.572 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=Hangzhou, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:36.572 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-11-29 08:44:36.572 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:36.606 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.607 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2025-11-29 08:44:36.608 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.609 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:44:36.610 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.611 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:44:36.612 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.613 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2025-11-29 08:44:36.613 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.614 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:44:36.615 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.616 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:44:36.617 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.618 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1004, ALTER_TENANT_LOCALITY] 2025-11-29 08:44:36.618 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.619 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1004, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:44:36.620 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.621 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1004, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:44:36.622 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2025-11-29 08:44:36.639 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.640 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1] 2025-11-29 08:44:36.643 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.644 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1010] 2025-11-29 08:44:36.644 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.202.22.58 2025-11-29 08:44:36.644 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:18.745924+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:36.644 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:36.673 INFO 12717 --- [Sync all tenant information1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=ocpserver, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:36.674 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:06.892896+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:36.674 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:36.675 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.675 INFO 12717 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE name = ?, args: [ob_tcp_invited_nodes] 2025-11-29 08:44:36.689 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:44:36.689 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2025-11-29 08:44:36.689 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=3, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:44:36.689 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=3, userWhitelist=% 2025-11-29 08:44:36.689 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:44:36.689 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2025-11-29 08:44:36.699 INFO 12717 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2025-11-29 08:44:36.699 INFO 12717 --- [Sync all tenant information1,,] .o.s.t.b.t.SyncAllTenantInfoShardingTask : [OCP606] sync tenants success, clusterId=1, clusterName=ocpserver, obClusterId=1743215593 2025-11-29 08:44:36.721 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.722 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1010] 2025-11-29 08:44:36.780 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.780 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1010] 2025-11-29 08:44:36.784 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.785 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1014] 2025-11-29 08:44:36.844 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:36.845 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1014] 2025-11-29 08:44:38.430 INFO 12717 --- [prometheus-minute-schedule-1,1b188aa360af0ed8,73534370474696f2] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:38.440 INFO 12717 --- [ocp-async-32,3f3bca29890120f7,fb15989583929dc5] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:38.462 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=5, ip=10.202.22.53, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T15:54:10.271583+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:38.462 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:30+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:38.463 INFO 12717 --- [prometheus-minute-schedule-1,1b188aa360af0ed8,73534370474696f2] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:38.508 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] slow query, durationMillis=1663, sql=SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ? 2025-11-29 08:44:38.509 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:38.510 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1014] 2025-11-29 08:44:38.516 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:38.516 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1018] 2025-11-29 08:44:38.580 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:38.581 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1018] 2025-11-29 08:44:38.644 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:38.644 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1018] 2025-11-29 08:44:38.648 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:38.649 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1020] 2025-11-29 08:44:38.714 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:38.715 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1020] 2025-11-29 08:44:38.720 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,3ef4ab1ad64e807a,ebc860415cac301c] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=3ef4ab1ad64e807a, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:38.721 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,3ef4ab1ad64e807a,ebc860415cac301c] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=3ef4ab1ad64e807a, duration=2 ms] 2025-11-29 08:44:38.726 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T16:19:29.514724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:38.726 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:38.738 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,35d31b80d5f54bb3,cffdab1aeabcfec9] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=35d31b80d5f54bb3, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:38.747 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,35d31b80d5f54bb3,cffdab1aeabcfec9] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=35d31b80d5f54bb3, duration=10 ms] 2025-11-29 08:44:38.754 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=6, ip=10.202.22.54, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=6, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-29T11:20:48.326103+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:38.754 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-11-29 08:44:38.777 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=5, name=zone_3, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:38.777 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:38.790 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:38.791 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1020] 2025-11-29 08:44:38.793 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:38.794 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1022] 2025-11-29 08:44:38.801 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=12, ip=10.202.22.48, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=12, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T12:28:53.231596+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:38.801 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:42+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:38.810 INFO 12717 --- [pool-18-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : Starting PLAN_CHANGE_DETECTOR, Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802} 2025-11-29 08:44:38.825 INFO 12717 --- [pool-18-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR : for [Pair(left=Tenant(id=14, name=pipeline, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-29T17:10:07+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1020, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=COMPLEX_OLTP), right=null), Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Pair(left=Tenant(id=11, name=oms, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T10:08:31+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1014, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTra 2025-11-29 08:44:38.829 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:44:38.831 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=13, ip=10.202.22.47, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=11, status=RUNNING, innerStatus=ACTIVE, startTime=2025-10-09T09:47:41.711435+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:38.831 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:07+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00} 2025-11-29 08:44:38.841 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=14, name=pipeline, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-29T17:10:07+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1020, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=COMPLEX_OLTP), right=null), Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:44:38.841 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=14, name=pipeline, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-29T17:10:07+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1020, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=COMPLEX_OLTP), right=null) 2025-11-29 08:44:38.841 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:44:38.857 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=11, ip=10.202.22.49, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=13, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T17:27:06.682649+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-11-29 08:44:38.861 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:38.861 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1022] 2025-11-29 08:44:38.879 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:38.880 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:38.881 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:44:38.888 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,e150c56576bf62e3,26025bd344a07aa1] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=e150c56576bf62e3, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:38.891 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,e150c56576bf62e3,26025bd344a07aa1] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=e150c56576bf62e3, duration=4 ms] 2025-11-29 08:44:38.892 WARN 12717 --- [host-check-clockdiff5,928390af0d622862,bb5b52fb9eca8843] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.53], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:38.892 ERROR 12717 --- [host-check-clockdiff5,928390af0d622862,bb5b52fb9eca8843] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.53] 2025-11-29 08:44:38.892 WARN 12717 --- [host-check-clockdiff5,928390af0d622862,bb5b52fb9eca8843] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.202.22.53, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.53 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:40.424 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.o.s.c.scheduled.TenantSchedules : Check suspend tenant compaction status 2025-11-29 08:44:40.424 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] s.t.b.t.CheckSuspendTenantCompactionTask : Check tenant compaction status. 2025-11-29 08:44:40.425 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] slow query, durationMillis=1543, sql=select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:44:40.425 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster DB 2025-11-29 08:44:40.425 INFO 12717 --- [ocp-schedules-8,dfb3446ff676c407,22ece1e257f792a8] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=2, successCount=2 2025-11-29 08:44:40.427 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:40.427 INFO 12717 --- [ocp-schedules-5,bf0d9a3013a65f8b,3647d81340f79b10] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:44:40.433 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:40.438 INFO 12717 --- [ocp-async-8,3a0e90eb5f202eec,14edc0d4c9e7ddfb] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:40.456 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] slow query, durationMillis=1595, sql=SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ? 2025-11-29 08:44:40.457 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:40.458 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1022] 2025-11-29 08:44:40.459 INFO 12717 --- [ocp-schedules-19,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=3 2025-11-29 08:44:40.460 INFO 12717 --- [ocp-schedules-8,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=3 2025-11-29 08:44:40.463 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:40.463 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1024] 2025-11-29 08:44:40.474 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 2025-11-29 08:44:40.482 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:44:40.492 INFO 12717 --- [Sync all clusters2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:44:40.499 INFO 12717 --- [ocp-schedules-2,2ea0374dc39b3421,4a37e54bd2e8d700] c.o.o.s.common.scheduled.PerfSchedules : Collect slow sql for alarm, time=2025-11-29T08:44:40.499+08:00, clusters=[1, 2] 2025-11-29 08:44:40.511 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:44:40.510484+08:00, lastProcessTime=2025-11-29T08:44:30.247713+08:00 2025-11-29 08:44:40.514 INFO 12717 --- [Check obproxy1,9d3c657eca6712fe,f7cebc68c58a1ec8] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:44:40.514 INFO 12717 --- [Check obproxy1,9d3c657eca6712fe,f7cebc68c58a1ec8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:40.514 INFO 12717 --- [Check obproxy3,435e6fad9c2ac9b7,c6f3e951a8e73238] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:44:40.514 INFO 12717 --- [Check obproxy3,435e6fad9c2ac9b7,c6f3e951a8e73238] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:40.517 INFO 12717 --- [Check obproxy2,a579eaa29a4ba947,70cbf7528020e4dd] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:44:40.517 INFO 12717 --- [Check obproxy2,a579eaa29a4ba947,70cbf7528020e4dd] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:40.517 INFO 12717 --- [Collect obproxy2,045ee2210760acb9,b401867aeb2e566e] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.52 3306 2025-11-29 08:44:40.517 INFO 12717 --- [Collect obproxy2,045ee2210760acb9,b401867aeb2e566e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:40.520 INFO 12717 --- [Collect obproxy3,9c98952c7bda59e8,d0a92cc5dc48b009] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.51 3306 2025-11-29 08:44:40.520 INFO 12717 --- [Collect obproxy3,9c98952c7bda59e8,d0a92cc5dc48b009] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:40.521 INFO 12717 --- [Collect obproxy1,8085f6d8f6d1e82e,e4f7e2655b95dd80] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 10.202.22.53 3306 2025-11-29 08:44:40.521 INFO 12717 --- [Collect obproxy1,8085f6d8f6d1e82e,e4f7e2655b95dd80] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-11-29 08:44:40.521 INFO 12717 --- [ocp-schedules-2,2ea0374dc39b3421,4a37e54bd2e8d700] c.o.o.p.s.i.alarm.SlowSqlDataSource : query sqlCount of obCluster ocpserver for slowSqlStat, sql count is 0. 2025-11-29 08:44:40.541 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:40.541 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:40.541 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1024] 2025-11-29 08:44:40.542 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-11-29 08:44:40.554 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290680554000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, sqlId=858015B92843B43D99CA77541C4C2592, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=14, obTenantId=1020, tenantName=pipeline, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:40.554Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=14, name=pipeline, obTenantId=1020, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort 2025-11-29 08:44:40.567 INFO 12717 --- [ocp-schedules-18,d16d80da521edafe,d07feef605bb09d6] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:44:40.618 INFO 12717 --- [Collect obproxy2,045ee2210760acb9,b401867aeb2e566e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:44:40.619 INFO 12717 --- [Check obproxy2,a579eaa29a4ba947,70cbf7528020e4dd] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:44:40.622 INFO 12717 --- [Check obproxy1,9d3c657eca6712fe,f7cebc68c58a1ec8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:44:40.623 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:40.623 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:40.623 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:40.623 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:40.625 INFO 12717 --- [Collect obproxy2,045ee2210760acb9,b401867aeb2e566e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:44:40.625 INFO 12717 --- [Check obproxy2,a579eaa29a4ba947,70cbf7528020e4dd] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-11-29 08:44:40.625 INFO 12717 --- [Check obproxy2,a579eaa29a4ba947,70cbf7528020e4dd] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 2, need to sync to meta: false 2025-11-29 08:44:40.627 INFO 12717 --- [Check obproxy1,9d3c657eca6712fe,f7cebc68c58a1ec8] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-11-29 08:44:40.627 INFO 12717 --- [Check obproxy1,9d3c657eca6712fe,f7cebc68c58a1ec8] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 1, need to sync to meta: false 2025-11-29 08:44:40.628 INFO 12717 --- [Collect obproxy2,045ee2210760acb9,b401867aeb2e566e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:44:40.634 INFO 12717 --- [ocp-schedules-18,d16d80da521edafe,d07feef605bb09d6] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:44:40.645 INFO 12717 --- [Check obproxy3,435e6fad9c2ac9b7,c6f3e951a8e73238] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-11-29 08:44:40.649 INFO 12717 --- [Collect obproxy1,8085f6d8f6d1e82e,e4f7e2655b95dd80] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:44:40.652 INFO 12717 --- [Check obproxy3,435e6fad9c2ac9b7,c6f3e951a8e73238] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2025-11-29 08:44:40.652 INFO 12717 --- [Check obproxy3,435e6fad9c2ac9b7,c6f3e951a8e73238] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 3 proxy_id: 3, need to sync to meta: false 2025-11-29 08:44:40.652 INFO 12717 --- [Collect obproxy3,9c98952c7bda59e8,d0a92cc5dc48b009] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-11-29 08:44:40.654 INFO 12717 --- [Collect obproxy1,8085f6d8f6d1e82e,e4f7e2655b95dd80] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:44:40.654 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1055 2025-11-29 08:44:40.654 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:40.656 INFO 12717 --- [Collect obproxy3,9c98952c7bda59e8,d0a92cc5dc48b009] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-11-29 08:44:40.657 INFO 12717 --- [Collect obproxy1,8085f6d8f6d1e82e,e4f7e2655b95dd80] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:44:40.659 INFO 12717 --- [Collect obproxy3,9c98952c7bda59e8,d0a92cc5dc48b009] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-11-29 08:44:40.661 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.50, port=2881, database=oceanbase) 2025-11-29 08:44:40.662 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.51, port=2881, database=oceanbase) 2025-11-29 08:44:40.662 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.52, port=2881, database=oceanbase) 2025-11-29 08:44:40.662 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=DB, obClusterId=1743215594, tenantName=sys, username=root, address=10.202.22.53, port=2881, database=oceanbase) 2025-11-29 08:44:40.662 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:40.663 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-11-29 08:44:40.669 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:40.670 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1024] 2025-11-29 08:44:40.675 INFO 12717 --- [ocp-schedules-25,bca80dc8a1ffbe3a,e595cc4b6dc51062] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:44:42.456 INFO 12717 --- [prometheus-minute-schedule-1,f21630ca99a74336,91f7b00331dd5d78] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:42.457 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,b83cbd6d3ed7aacb,508d8ea0fc385d21] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=b83cbd6d3ed7aacb, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:42.457 WARN 12717 --- [host-check-clockdiff3,928390af0d622862,607e0c693c2ef1b4] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.51], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:42.457 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,b83cbd6d3ed7aacb,508d8ea0fc385d21] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=b83cbd6d3ed7aacb, duration=1 ms] 2025-11-29 08:44:42.457 ERROR 12717 --- [host-check-clockdiff3,928390af0d622862,607e0c693c2ef1b4] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.51] 2025-11-29 08:44:42.457 WARN 12717 --- [host-check-clockdiff3,928390af0d622862,607e0c693c2ef1b4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.202.22.51, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.51 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:42.458 INFO 12717 --- [ocp-async-1,d5fe3689981a12d0,6f039005e7b4ca5b] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:42.463 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:42.463 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:42.464 INFO 12717 --- [Sync all clusters2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-11-29 08:44:42.466 INFO 12717 --- [ocp-schedules-1,a1df35f98b397ab8,4946e3571d70c195] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] slow query, durationMillis=1802, sql=SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-11-29 08:44:42.466 WARN 12717 --- [ocp-updater-0,,] c.o.o.c.d.service.DefaultServiceUpdater : Deleted expired service: DistributedService{name=PLAN_CHANGE_DETECTOR, serverId=10.202.22.58:8080, expireInSeconds=30, createTime=2025-11-29T08:34:41+08:00, updateTime=2025-11-29T08:44:11+08:00, expireTime=null, isLeader=false} 2025-11-29 08:44:42.469 INFO 12717 --- [prometheus-minute-schedule-1,f21630ca99a74336,91f7b00331dd5d78] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:42.475 WARN 12717 --- [ocp-updater-0,,] c.o.o.c.d.service.DefaultServiceUpdater : No leader or more than one leader exists for service PLAN_CHANGE_DETECTOR, Incorrect result size: expected 1, actual 0 2025-11-29 08:44:42.489 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.490 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2025-11-29 08:44:42.492 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.493 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:44:42.494 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:42.494 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:42.495 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.496 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:44:42.498 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.499 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1010, ALTER_TENANT_LOCALITY] 2025-11-29 08:44:42.502 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.503 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1010, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:44:42.505 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.506 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1010, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:44:42.509 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.509 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1014, ALTER_TENANT_LOCALITY] 2025-11-29 08:44:42.512 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.513 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1014, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:44:42.516 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.517 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:42.517 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1020, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:42.517 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1014, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:44:42.520 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.521 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1018, ALTER_TENANT_LOCALITY] 2025-11-29 08:44:42.522 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:44:42.524 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.525 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1018, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:44:42.527 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.528 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1018, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:44:42.532 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.533 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1020, ALTER_TENANT_LOCALITY] 2025-11-29 08:44:42.536 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.537 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1020, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:44:42.541 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.542 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1020, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:44:42.546 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.546 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1022, ALTER_TENANT_LOCALITY] 2025-11-29 08:44:42.549 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.549 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1022, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:44:42.550 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:40.554, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=14, sqlId=858015B92843B43D99CA77541C4C2592, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:42.551 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:44:42.551 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.552 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1022, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:44:42.554 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:44:42.554 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null) 2025-11-29 08:44:42.554 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:44:42.554 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.555 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1024, ALTER_TENANT_LOCALITY] 2025-11-29 08:44:42.557 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.558 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1024, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-11-29 08:44:42.560 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.561 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1024, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-11-29 08:44:42.563 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=2 2025-11-29 08:44:42.568 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.202.22.58 2025-11-29 08:44:42.605 INFO 12717 --- [Sync all tenant information2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:42.606 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.607 INFO 12717 --- [Sync all tenant information2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE name = ?, args: [ob_tcp_invited_nodes] 2025-11-29 08:44:42.610 INFO 12717 --- [Sync all clusters2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 2, sqlPort: 2881 got ObProcessInfoResult(pid=2447, runPath=/data/oceanbase, runUser=admin, execute=/data/oceanbase/bin/observer, installPath=/data/oceanbase) 2025-11-29 08:44:42.628 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=DB, tenantName=sys, dbUser=ocp_monitor 2025-11-29 08:44:42.629 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:42.630 INFO 12717 --- [Sync all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-11-29 08:44:42.643 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682643000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, sqlId=4E652C54963ADF0C0155B0838CC3F9C2, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=9, obTenantId=1010, tenantName=lhp6796, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.643Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=9, name=lhp6796, obTenantId=1010, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=288 2025-11-29 08:44:42.647 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682647000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, sqlId=858015B92843B43D99CA77541C4C2592, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=9, obTenantId=1010, tenantName=lhp6796, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.647Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=9, name=lhp6796, obTenantId=1010, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=288 2025-11-29 08:44:42.648 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682648000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, sqlId=AD4CCCEE430EDC50CC9CBDC52C24827E, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=9, obTenantId=1010, tenantName=lhp6796, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.648Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=9, name=lhp6796, obTenantId=1010, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=288 2025-11-29 08:44:42.653 INFO 12717 --- [Sync all clusters2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=DB, obClusterId=1743215594 2025-11-29 08:44:42.653 INFO 12717 --- [Sync all clusters2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=DB, obClusterId=1743215594, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=CREATED, syncStatus=VALID, rootserverJson={"ObCluster":"DB","ObClusterId":1743215594,"ObRegion":"DB","ObRegionId":1743215594,"ReadonlyRsList":[],"RsList":[{"address":"10.202.22.52:2882","role":"LEADER","sql_port":2881},{"address":"10.202.22.49:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.202.22.61:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1764376629943000}, createTime=2025-03-29T11:15:04+08:00, creator=admin, obVersion=4.3.5.1, packageOperatingSystem=el8, loadType=HTAP, tenantCount=7) 2025-11-29 08:44:42.657 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 48 2025-11-29 08:44:42.657 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:42.657 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 48 2025-11-29 08:44:42.658 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=9, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=9, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=13, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=13, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=11, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=11, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=14, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=14, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=4, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=4, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=16, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=16, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=15, realWhitelist=%, defaultWhitelist=127.0.0.1,10.202.22.58, userWhitelist=% 2025-11-29 08:44:42.663 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=15, userWhitelist=% 2025-11-29 08:44:42.672 INFO 12717 --- [Sync all tenant information2,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=2 2025-11-29 08:44:42.672 INFO 12717 --- [Sync all tenant information2,,] .o.s.t.b.t.SyncAllTenantInfoShardingTask : [OCP606] sync tenants success, clusterId=2, clusterName=DB, obClusterId=1743215594 2025-11-29 08:44:42.675 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:42.675 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=AD4CCCEE430EDC50CC9CBDC52C24827E, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:42.676 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:42.676 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=4E652C54963ADF0C0155B0838CC3F9C2, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:42.684 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:42.684 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:42.685 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:42.685 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:42.693 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1043 2025-11-29 08:44:42.693 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:42.704 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:42.704 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=4E652C54963ADF0C0155B0838CC3F9C2, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:42.704 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:42.704 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=AD4CCCEE430EDC50CC9CBDC52C24827E, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:42.708 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:44:42.709 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:44:42.716 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:42.716 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:42.723 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:42.723 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:42.724 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.648, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=9, sqlId=AD4CCCEE430EDC50CC9CBDC52C24827E, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:42.743 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:44:40.510484+08:00 2025-11-29 08:44:42.745 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.643, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=9, sqlId=4E652C54963ADF0C0155B0838CC3F9C2, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:42.745 INFO 12717 --- [ocp-schedules-2,2ea0374dc39b3421,4a37e54bd2e8d700] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for DB, save 30 to SqlAggregateData 2025-11-29 08:44:42.745 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:42.745 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1010, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:42.751 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:44:42.770 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.647, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=9, sqlId=858015B92843B43D99CA77541C4C2592, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:42.771 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:44:42.774 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=11, name=oms, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T10:08:31+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1014, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:44:42.774 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=11, name=oms, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T10:08:31+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1014, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null) 2025-11-29 08:44:42.774 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:44:42.847 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682847000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=9168B12FC00A9008C84CE9A336E27A96, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.847Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:42.849 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682849000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=5B16335FD7B1D26B67825C0F995123BE, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.849Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:42.849 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682849000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=6C06063F61D7F82604EA25EE26C107F5, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.849Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:42.850 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682850000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=CB73A583C8E0474BC3A6A5C4578F229D, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.850Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:42.851 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682851000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=858015B92843B43D99CA77541C4C2592, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.851Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:42.852 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682852000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=D9EB4937E65F94AA38F6B0FE72B06D13, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.852Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:42.855 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682855000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=5597EAE1A24B2234E36E41AC54209C74, obServerId=null, obDbId=500004, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.855Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:42.863 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290682863000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=BF8B77776E4E25AE094BB00E6E2FA473, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:42.863Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:42.872 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 265 2025-11-29 08:44:42.872 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:42.882 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 535 2025-11-29 08:44:42.882 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:42.885 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 696 2025-11-29 08:44:42.885 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:42.889 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 262 2025-11-29 08:44:42.889 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:42.897 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:42.897 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=9168B12FC00A9008C84CE9A336E27A96, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.423 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:44.425 INFO 12717 --- [prometheus-minute-schedule-1,79b146c83d8a539f,e296f1c08d4571ae] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:44.427 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,24e52d9e03891f74,1bad5bcaaa81677f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=24e52d9e03891f74, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:44.428 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,24e52d9e03891f74,1bad5bcaaa81677f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=24e52d9e03891f74, duration=2 ms] 2025-11-29 08:44:44.430 INFO 12717 --- [ocp-async-32,d0babcfe841ee2a3,dc60f4dde7918001] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:44:44.431 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2025-11-29 08:44:44.431 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:44.436 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:44.436 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:44.439 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1050 2025-11-29 08:44:44.439 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:44.442 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 531 2025-11-29 08:44:44.442 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:44.443 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1195 2025-11-29 08:44:44.443 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:44.443 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:44.459 INFO 12717 --- [ocp-async-37,f1fd13de011187cd,971d2239c23eb05b] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:44.459 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.459 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=5B16335FD7B1D26B67825C0F995123BE, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.466 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.466 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=6C06063F61D7F82604EA25EE26C107F5, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.476 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.476 INFO 12717 --- [prometheus-minute-schedule-1,79b146c83d8a539f,e296f1c08d4571ae] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:44.476 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=CB73A583C8E0474BC3A6A5C4578F229D, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.491 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.491 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=201001, sqlId=D9EB4937E65F94AA38F6B0FE72B06D13, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.492 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.492 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500004, sqlId=5597EAE1A24B2234E36E41AC54209C74, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.492 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.492 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.494 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.494 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=9168B12FC00A9008C84CE9A336E27A96, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.495 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.495 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=BF8B77776E4E25AE094BB00E6E2FA473, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.563 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:44.563 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:44.565 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:44.565 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:44.573 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:44.573 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:44.575 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:44:44.576 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:44.576 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:44.577 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:44.577 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:44.579 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:44.579 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:44.582 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:44.582 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:44.591 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.591 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=5B16335FD7B1D26B67825C0F995123BE, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.594 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.594 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=6C06063F61D7F82604EA25EE26C107F5, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.595 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:44:44.595 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.847, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=11, sqlId=9168B12FC00A9008C84CE9A336E27A96, serverId=null, dbName=_cm, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:44.599 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:44:44.599 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.599 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=201001, sqlId=D9EB4937E65F94AA38F6B0FE72B06D13, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.600 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.600 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.602 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.602 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500004, sqlId=5597EAE1A24B2234E36E41AC54209C74, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.602 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.603 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=BF8B77776E4E25AE094BB00E6E2FA473, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.605 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.605 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500002, sqlId=CB73A583C8E0474BC3A6A5C4578F229D, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.606 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:44:44.606 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:44:44.607 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:44:44.608 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:44:44.610 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:44:44.627 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.855, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=11, sqlId=5597EAE1A24B2234E36E41AC54209C74, serverId=null, dbName=_cm_hb, obDbId=500004, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:44.628 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.849, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=11, sqlId=5B16335FD7B1D26B67825C0F995123BE, serverId=null, dbName=_cm, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:44.631 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.85, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=11, sqlId=CB73A583C8E0474BC3A6A5C4578F229D, serverId=null, dbName=_cm, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:44.634 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.849, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=11, sqlId=6C06063F61D7F82604EA25EE26C107F5, serverId=null, dbName=_cm, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:44.638 INFO 12717 --- [plan_change_detector32,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.852, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=11, sqlId=D9EB4937E65F94AA38F6B0FE72B06D13, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:44.639 INFO 12717 --- [plan_change_detector31,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.863, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=11, sqlId=BF8B77776E4E25AE094BB00E6E2FA473, serverId=null, dbName=_cm, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:44.641 INFO 12717 --- [plan_change_detector29,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:42.851, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=11, sqlId=858015B92843B43D99CA77541C4C2592, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:44.662 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290684661000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, sqlId=08DA9D32423475AF5D841F69CAC1F7B8, obServerId=null, obDbId=500003, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=11, obTenantId=1014, tenantName=oms, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:44.661Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:44.680 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 260 2025-11-29 08:44:44.681 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:44.698 INFO 12717 --- [ocp-schedules-20,4764c6564a9ad8f0,81fcf48dd1f09728] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:44:44.699 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.699 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500003, sqlId=08DA9D32423475AF5D841F69CAC1F7B8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.704 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:44.704 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:44.712 INFO 12717 --- [alarm-detect-11,4764c6564a9ad8f0,2d1eb1d84e1d9cc4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008342, data is empty. 2025-11-29 08:44:44.712 INFO 12717 --- [alarm-detect-6,4764c6564a9ad8f0,38f1da89b8cc2dda] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008342, data is empty. 2025-11-29 08:44:44.713 INFO 12717 --- [alarm-detect-9,4764c6564a9ad8f0,7f244e95eec2106b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008342, data is empty. 2025-11-29 08:44:44.720 INFO 12717 --- [alarm-detect-11,4764c6564a9ad8f0,2d1eb1d84e1d9cc4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008372, data is empty. 2025-11-29 08:44:44.720 INFO 12717 --- [alarm-detect-6,4764c6564a9ad8f0,38f1da89b8cc2dda] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008372, data is empty. 2025-11-29 08:44:44.721 INFO 12717 --- [alarm-detect-9,4764c6564a9ad8f0,7f244e95eec2106b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008372, data is empty. 2025-11-29 08:44:44.731 WARN 12717 --- [scanner-11213818319-1,4764c6564a9ad8f0,b9c87e82c39c3a10] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:44:44.732 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:44.732 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1014, obServerId=null, obDbId=500003, sqlId=08DA9D32423475AF5D841F69CAC1F7B8, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:44.736 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-11-29 08:44:44.746 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:44.661, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=11, sqlId=08DA9D32423475AF5D841F69CAC1F7B8, serverId=null, dbName=_rm, obDbId=500003, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:44.748 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:44:44.755 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=13, name=nextcloud, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T13:13:08+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1018, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:44:44.757 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=13, name=nextcloud, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-02T13:13:08+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1018, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null) 2025-11-29 08:44:44.757 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:44:44.773 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:44:44.777 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=2, name=ocp_meta, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:44:44.777 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=2, name=ocp_meta, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null) 2025-11-29 08:44:44.777 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:44:44.787 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:44:44.789 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=3, name=ocp_monitor, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1004, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:44:44.790 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=3, name=ocp_monitor, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1004, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null) 2025-11-29 08:44:44.790 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:44:44.790 INFO 12717 --- [alarm-detect-9,4764c6564a9ad8f0,beb491c41f41b71a] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008342, data is empty. 2025-11-29 08:44:44.794 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:44:44.797 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=16, name=wecom_bot, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-27T13:42:13+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1024, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:44:44.797 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=16, name=wecom_bot, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-27T13:42:13+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1024, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null) 2025-11-29 08:44:44.797 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:44:44.798 INFO 12717 --- [alarm-detect-9,4764c6564a9ad8f0,beb491c41f41b71a] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008372, data is empty. 2025-11-29 08:44:44.864 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290684864000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, sqlId=858015B92843B43D99CA77541C4C2592, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=16, obTenantId=1024, tenantName=wecom_bot, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:44.864Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=16, name=wecom_bot, obTenantId=1024, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPo 2025-11-29 08:44:46.515 INFO 12717 --- [ocp-async-21,ab78c6770692220c,ee4ceb0962927cae] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:46.516 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,4e46b2a0f2124dfe,7eb1ccc790dc00e4] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=4e46b2a0f2124dfe, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:46.525 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1041 2025-11-29 08:44:46.525 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:46.527 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,4e46b2a0f2124dfe,7eb1ccc790dc00e4] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=4e46b2a0f2124dfe, duration=13 ms] 2025-11-29 08:44:46.552 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:46.552 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:46.559 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:46.559 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:46.565 WARN 12717 --- [host-check-clockdiff10,928390af0d622862,cbbdddb5a2636627] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.47], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:46.565 ERROR 12717 --- [host-check-clockdiff10,928390af0d622862,cbbdddb5a2636627] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.47] 2025-11-29 08:44:46.566 WARN 12717 --- [host-check-clockdiff10,928390af0d622862,cbbdddb5a2636627] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=11, ip=10.202.22.47, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.47 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:46.577 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:46.577 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1024, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:46.582 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:44:46.601 INFO 12717 --- [plan_change_detector10,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:44.864, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=16, sqlId=858015B92843B43D99CA77541C4C2592, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:46.602 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:44:46.605 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=15, name=zto, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-06-19T19:31:51+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1022, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:44:46.605 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=15, name=zto, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-06-19T19:31:51+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1022, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null) 2025-11-29 08:44:46.605 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:44:46.683 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290686683000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, sqlId=A46E6C33C9B9103C8D102613F60328F7, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=15, obTenantId=1022, tenantName=zto, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:46.683Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=15, name=zto, obTenantId=1022, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:46.683 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290686683000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, sqlId=858015B92843B43D99CA77541C4C2592, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=15, obTenantId=1022, tenantName=zto, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:46.683Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=15, name=zto, obTenantId=1022, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:46.686 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290686686000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, sqlId=CBE25078740DF69A7D871A200E98EBBD, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=15, obTenantId=1022, tenantName=zto, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:46.686Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=15, name=zto, obTenantId=1022, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, ins 2025-11-29 08:44:46.687 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 0 2025-11-29 08:44:46.687 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:46.690 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 0 2025-11-29 08:44:46.690 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:46.707 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:46.707 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=500002, sqlId=A46E6C33C9B9103C8D102613F60328F7, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:46.709 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:46.709 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=500002, sqlId=CBE25078740DF69A7D871A200E98EBBD, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:46.711 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:46.711 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:46.713 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:46.713 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:46.729 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:46.729 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=500002, sqlId=A46E6C33C9B9103C8D102613F60328F7, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:46.732 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:46.732 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=500002, sqlId=CBE25078740DF69A7D871A200E98EBBD, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:46.732 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records,use plan union hash 2025-11-29 08:44:46.732 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 0 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:46.683, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=15, sqlId=A46E6C33C9B9103C8D102613F60328F7, serverId=null, dbName=networkbasedata, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:46.734 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records,use plan union hash 2025-11-29 08:44:46.734 INFO 12717 --- [plan_change_detector24,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 0 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:46.686, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=15, sqlId=CBE25078740DF69A7D871A200E98EBBD, serverId=null, dbName=networkbasedata, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:46.739 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 1519 2025-11-29 08:44:46.739 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:46.759 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:46.759 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:46.765 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:46.765 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:46.785 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:46.785 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1022, obServerId=null, obDbId=201001, sqlId=858015B92843B43D99CA77541C4C2592, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:46.789 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-11-29 08:44:46.806 INFO 12717 --- [plan_change_detector25,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:46.683, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=15, sqlId=858015B92843B43D99CA77541C4C2592, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:46.807 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:44:46.809 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:44:46.809 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=1, name=sys, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null) 2025-11-29 08:44:46.809 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:44:46.811 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-11-29 08:44:46.816 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=4, name=sys, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T11:20:46+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-11-29T08:40:36.591, end=2025-11-29T08:42:38.802},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-11-29 08:44:46.816 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=4, name=sys, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T11:20:46+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null) 2025-11-29 08:44:46.816 INFO 12717 --- [pool-18-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-11-29 08:44:46.900 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290686899000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1, sqlId=B1A1F3E7428A06C68C941CE70F77CEEE, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=4, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:46.899Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=4, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/d 2025-11-29 08:44:46.900 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1764290686900000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1, sqlId=D66031DC458A041BD89D3AFB4937145E, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=2, obClusterId=1743215594, clusterName=DB, tenantId=4, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.1, startTime=2025-11-28T00:44:46.900Z, endTime=2025-11-29T00:42:38.802Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], serverMap={1=10.202.22.50:2882, 2=10.202.22.51:2882, 3=10.202.22.52:2882, 4=10.202.22.53:2882, 5=10.202.22.54:2882, 6=10.202.22.59:2882, 7=10.202.22.60:2882, 8=10.202.22.61:2882, 9=10.202.22.49:2882, 10=10.202.22.48:2882, 11=10.202.22.47:2882}, serverIpMap={10.202.22.59=6, 10.202.22.48=10, 10.202.22.47=11, 10.202.22.49=9, 10.202.22.60=7, 10.202.22.51=2, 10.202.22.50=1, 10.202.22.61=8, 10.202.22.53=4, 10.202.22.52=3, 10.202.22.54=5}, cluster=ObClusterEntity{id=2, name=DB, obClusterId=1743215594, status=RUNNING}, tenant=ObTenantEntity{id=4, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=2, ip=10.202.22.50, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:41:07+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.202.22.51, port=2882, sqlPort=2881, installPath=/data/oceanbase, runPath=/data/oceanbase, runUser=admin, version=4.3.5.1_101000042025031818-b6d5706eb3d2c5f501c7fa646ddbf32f3dc87069(Mar 18 2025 18:13:36), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-03-31T13:56:19+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=4, ip=10.202.22.52, port=2882, sqlPort=2881, installPath=/d 2025-11-29 08:44:48.528 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:48.529 INFO 12717 --- [prometheus-minute-schedule-1,5f94b08ee7d2fe45,33aba4639e4734e6] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:48.530 INFO 12717 --- [ocp-schedules-17,dd7311e9c90f5076,35de5e9b3c5a9560] c.o.o.s.c.scheduled.TenantSchedules : Collect all tenant major compaction 2025-11-29 08:44:48.530 INFO 12717 --- [ocp-schedules-17,dd7311e9c90f5076,35de5e9b3c5a9560] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2025-11-29 08:44:48.530 INFO 12717 --- [ocp-async-33,eccafba9dc727483,43f982627261abbf] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:48.531 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,650d5e5fd3da3d2d,e76bcacd9ecf5d28] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=650d5e5fd3da3d2d, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:48.531 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-78,650d5e5fd3da3d2d,e76bcacd9ecf5d28] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=650d5e5fd3da3d2d, duration=1 ms] 2025-11-29 08:44:48.538 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:48.543 INFO 12717 --- [ocp-schedules-2,74c8945eafb783b2,4f3e2cb23754e876] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-11-29 08:44:48.552 INFO 12717 --- [prometheus-minute-schedule-1,5f94b08ee7d2fe45,33aba4639e4734e6] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:48.555 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 2609 2025-11-29 08:44:48.555 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:48.556 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 2609 2025-11-29 08:44:48.556 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-11-29 08:44:48.588 INFO 12717 --- [ocp-schedules-2,74c8945eafb783b2,4f3e2cb23754e876] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:44:48.588 INFO 12717 --- [ocp-schedules-2,74c8945eafb783b2,4f3e2cb23754e876] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[206790b4-b3a1-4b5c-813c-b9e29360c26d] 2025-11-29 08:44:48.602 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:48.602 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1, obServerId=null, obDbId=201001, sqlId=B1A1F3E7428A06C68C941CE70F77CEEE, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:48.604 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:48.604 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1, obServerId=null, obDbId=201001, sqlId=D66031DC458A041BD89D3AFB4937145E, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:48.614 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:48.614 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:48.614 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-11-29 08:44:48.614 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-11-29 08:44:48.617 INFO 12717 --- [Collect all tenants major compactions3,dd7311e9c90f5076,646df7f72f93fcd6] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:sys 2025-11-29 08:44:48.617 INFO 12717 --- [Collect all tenants major compactions6,dd7311e9c90f5076,bb59cd5f9562a80a] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:oms 2025-11-29 08:44:48.617 INFO 12717 --- [Collect all tenants major compactions2,dd7311e9c90f5076,45f407a217ea5e01] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:ocp_monitor 2025-11-29 08:44:48.617 INFO 12717 --- [Collect all tenants major compactions8,dd7311e9c90f5076,853c4f57acead43b] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:sys 2025-11-29 08:44:48.618 INFO 12717 --- [Collect all tenants major compactions1,dd7311e9c90f5076,6720c30cb676a969] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:ocpserver, tenant:ocp_meta 2025-11-29 08:44:48.618 INFO 12717 --- [Collect all tenants major compactions9,dd7311e9c90f5076,6c0e74f6c7bd9335] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:wecom_bot 2025-11-29 08:44:48.618 INFO 12717 --- [Collect all tenants major compactions5,dd7311e9c90f5076,268f26b7d95867ce] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:nextcloud 2025-11-29 08:44:48.618 INFO 12717 --- [Collect all tenants major compactions4,dd7311e9c90f5076,c687725a192584cd] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:lhp6796 2025-11-29 08:44:48.618 INFO 12717 --- [Collect all tenants major compactions10,dd7311e9c90f5076,22ffdc6afcbaf9b4] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:zto 2025-11-29 08:44:48.619 INFO 12717 --- [Collect all tenants major compactions7,dd7311e9c90f5076,14613aaf6e26f856] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:DB, tenant:pipeline 2025-11-29 08:44:48.651 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:48.651 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1, obServerId=null, obDbId=201001, sqlId=B1A1F3E7428A06C68C941CE70F77CEEE, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:48.651 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-11-29 08:44:48.651 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1764288000000000, endTimeUs=1764376958802000, obClusterId=1743215594, clusterName=DB, obTenantId=1, obServerId=null, obDbId=201001, sqlId=D66031DC458A041BD89D3AFB4937145E, sqlIdList=null, dynamicSql=false) 2025-11-29 08:44:48.654 INFO 12717 --- [Collect all tenants major compactions2,dd7311e9c90f5076,45f407a217ea5e01] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:48.654 INFO 12717 --- [Collect all tenants major compactions3,dd7311e9c90f5076,646df7f72f93fcd6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:48.655 INFO 12717 --- [Collect all tenants major compactions2,dd7311e9c90f5076,45f407a217ea5e01] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1004] 2025-11-29 08:44:48.655 INFO 12717 --- [Collect all tenants major compactions5,dd7311e9c90f5076,268f26b7d95867ce] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:48.655 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 10 records,use plan union hash 2025-11-29 08:44:48.655 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 10 records,use plan union hash 2025-11-29 08:44:48.656 INFO 12717 --- [Collect all tenants major compactions8,dd7311e9c90f5076,853c4f57acead43b] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:48.656 INFO 12717 --- [Collect all tenants major compactions4,dd7311e9c90f5076,c687725a192584cd] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:48.656 INFO 12717 --- [Collect all tenants major compactions10,dd7311e9c90f5076,22ffdc6afcbaf9b4] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:48.656 INFO 12717 --- [Collect all tenants major compactions6,dd7311e9c90f5076,bb59cd5f9562a80a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:48.656 INFO 12717 --- [Collect all tenants major compactions3,dd7311e9c90f5076,646df7f72f93fcd6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-11-29 08:44:48.656 INFO 12717 --- [Collect all tenants major compactions7,dd7311e9c90f5076,14613aaf6e26f856] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:48.656 INFO 12717 --- [Collect all tenants major compactions9,dd7311e9c90f5076,6c0e74f6c7bd9335] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:48.657 INFO 12717 --- [Collect all tenants major compactions8,dd7311e9c90f5076,853c4f57acead43b] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-11-29 08:44:48.657 INFO 12717 --- [Collect all tenants major compactions5,dd7311e9c90f5076,268f26b7d95867ce] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1018] 2025-11-29 08:44:48.658 INFO 12717 --- [Collect all tenants major compactions4,dd7311e9c90f5076,c687725a192584cd] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1010] 2025-11-29 08:44:48.658 INFO 12717 --- [Collect all tenants major compactions6,dd7311e9c90f5076,bb59cd5f9562a80a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1014] 2025-11-29 08:44:48.658 INFO 12717 --- [Collect all tenants major compactions1,dd7311e9c90f5076,6720c30cb676a969] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:44:48.658 INFO 12717 --- [Collect all tenants major compactions7,dd7311e9c90f5076,14613aaf6e26f856] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1020] 2025-11-29 08:44:48.659 INFO 12717 --- [Collect all tenants major compactions9,dd7311e9c90f5076,6c0e74f6c7bd9335] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1024] 2025-11-29 08:44:48.659 INFO 12717 --- [Collect all tenants major compactions10,dd7311e9c90f5076,22ffdc6afcbaf9b4] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1022] 2025-11-29 08:44:48.659 INFO 12717 --- [Collect all tenants major compactions1,dd7311e9c90f5076,6720c30cb676a969] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2025-11-29 08:44:48.661 INFO 12717 --- [Collect all tenants major compactions2,dd7311e9c90f5076,45f407a217ea5e01] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:ocp_monitor 2025-11-29 08:44:48.661 INFO 12717 --- [Collect all tenants major compactions3,dd7311e9c90f5076,646df7f72f93fcd6] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:sys 2025-11-29 08:44:48.662 INFO 12717 --- [Collect all tenants major compactions1,dd7311e9c90f5076,6720c30cb676a969] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:ocpserver, tenant:ocp_meta 2025-11-29 08:44:48.662 INFO 12717 --- [Collect all tenants major compactions7,dd7311e9c90f5076,14613aaf6e26f856] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:pipeline 2025-11-29 08:44:48.663 INFO 12717 --- [Collect all tenants major compactions6,dd7311e9c90f5076,bb59cd5f9562a80a] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:oms 2025-11-29 08:44:48.664 INFO 12717 --- [Collect all tenants major compactions8,dd7311e9c90f5076,853c4f57acead43b] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:sys 2025-11-29 08:44:48.664 INFO 12717 --- [Collect all tenants major compactions4,dd7311e9c90f5076,c687725a192584cd] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:lhp6796 2025-11-29 08:44:48.664 INFO 12717 --- [Collect all tenants major compactions5,dd7311e9c90f5076,268f26b7d95867ce] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:nextcloud 2025-11-29 08:44:48.664 INFO 12717 --- [Collect all tenants major compactions10,dd7311e9c90f5076,22ffdc6afcbaf9b4] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:zto 2025-11-29 08:44:48.664 INFO 12717 --- [Collect all tenants major compactions9,dd7311e9c90f5076,6c0e74f6c7bd9335] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:DB, tenant:wecom_bot 2025-11-29 08:44:48.666 INFO 12717 --- [ocp-schedules-17,dd7311e9c90f5076,35de5e9b3c5a9560] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2025-11-29 08:44:48.710 INFO 12717 --- [plan_change_detector22,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:46.899, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=4, sqlId=B1A1F3E7428A06C68C941CE70F77CEEE, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:48.723 INFO 12717 --- [plan_change_detector28,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-11-28T08:44:46.9, end=2025-11-29T08:42:38.802}, clusterId=2, tenantId=4, sqlId=D66031DC458A041BD89D3AFB4937145E, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-11-29 08:44:48.723 INFO 12717 --- [pool-18-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR, Found out 0 dubious sqls 2025-11-29 08:44:50.527 INFO 12717 --- [ocp-schedules-9,67fa7eb9e94e07fc,4c59ccf67cb48a7a] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:44:50.527 INFO 12717 --- [prometheus-minute-schedule-1,5236657091096834,7d2534f1e2a75205] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:50.562 WARN 12717 --- [host-check-clockdiff2,928390af0d622862,0583236f193e0ee3] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.50], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 143 (Exit value: 143) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:50.562 ERROR 12717 --- [host-check-clockdiff2,928390af0d622862,0583236f193e0ee3] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.50] 2025-11-29 08:44:50.562 WARN 12717 --- [host-check-clockdiff2,928390af0d622862,0583236f193e0ee3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.202.22.50, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.50 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:50.562 WARN 12717 --- [host-check-clockdiff7,928390af0d622862,12b8c6dc687f5ea2] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.59], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 143 (Exit value: 143) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:50.562 ERROR 12717 --- [host-check-clockdiff7,928390af0d622862,12b8c6dc687f5ea2] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.59] 2025-11-29 08:44:50.562 WARN 12717 --- [host-check-clockdiff7,928390af0d622862,12b8c6dc687f5ea2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=8, ip=10.202.22.59, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.59 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:50.563 WARN 12717 --- [host-check-clockdiff8,928390af0d622862,69394cd8d20fa6e5] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.60], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 143 (Exit value: 143) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:50.563 ERROR 12717 --- [host-check-clockdiff8,928390af0d622862,69394cd8d20fa6e5] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.60] 2025-11-29 08:44:50.563 WARN 12717 --- [host-check-clockdiff8,928390af0d622862,69394cd8d20fa6e5] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=9, ip=10.202.22.60, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.60 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:50.563 WARN 12717 --- [host-check-clockdiff9,928390af0d622862,cb4ab887246535a6] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.61], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 143 (Exit value: 143) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:50.563 ERROR 12717 --- [host-check-clockdiff9,928390af0d622862,cb4ab887246535a6] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.61] 2025-11-29 08:44:50.563 WARN 12717 --- [host-check-clockdiff9,928390af0d622862,cb4ab887246535a6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=10, ip=10.202.22.61, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.61 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:50.566 INFO 12717 --- [ocp-async-22,2f2bc4a947eac3c7,4062ce376c4fe427] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:50.566 INFO 12717 --- [prometheus-cache-monitor1,3e1ebebce9ac6988,51bb7ad6686385cf] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":40354,"minuteWriteQueue":0,"minuteReadCacheSize":17215,"secondWriteCacheSize":10091,"seriesIdKeySize":17387,"secondWriteQueue":0,"minuteWriteCacheSize":17216} 2025-11-29 08:44:50.566 INFO 12717 --- [prometheus-cache-monitor1,3e1ebebce9ac6988,51bb7ad6686385cf] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{ob_sysstat_40012=49, ob_server_resource_memory_assigned_bytes=23, ob_sysstat_40011=49, ob_sysstat_40010=49, ob_unit_num=23, ob_database_disk_required_size=74, go_memstats_mspan_sys_bytes=25, ob_sysstat_30213=42, go_memstats_stack_sys_bytes=25, ob_sysstat_30212=42, ob_sysstat_30217=42, ob_sysstat_30216=42, ob_tenant_request_queue_total_size=72, jvm_gc_memory_allocated_bytes_total=2, ob_system_event_total_waits=59, ob_tenant_disk_v4_log_disk_size=40, ob_sysstat_40019=49, ob_sysstat_40018=49, processor_process_metrics_total=30, jvm_gc_max_data_size_bytes=2, go_memstats_frees_total=25, host_available=29, ob_sysstat_30220=42, ob_sysstat_30222=42, ob_sysstat_30221=42, mysql_output_write_sql_failed_count=29, ob_database_disk_data_size=73, ob_tenant_memstore_freeze_cnt=40, ob_tenant_unit_min_memory=96, ob_sysstat_60087=49, ob_memstore_total_bytes=72, jvm_threads_states_threads=8, process_resident_memory_bytes=25, ob_cache_size_bytes=67, ob_tenant_unit_max_memory=96, input_collect_metrics_total=34, ob_sysstat_50039=50, ob_sysstat_60083=50, ob_sysstat_50038=49, node_disk_io_time_weighted_seconds_total=39, process_open_fds=25, ob_sysstat_50037=49, process_memory_rss_bytes=41, ob_server_resource_cpu_assigned_percent=23, ob_sysstat_50030=50, sql_audit_input_collect_lost_total=36, go_memstats_heap_inuse_bytes=25, ob_sysstat_40118=49, ob_sysstat_40117=49, node_memory_memfree_bytes=32, ob_sysstat_40116=49, ob_sysstat_40005=49, process_exists=50, ob_sysstat_40004=49, ob_sysstat_40003=49, ob_sysstat_40002=49, go_gc_duration_seconds=30, ob_server_resource_disk_bytes=23, ob_sysstat_40001=49, ob_sysstat_40000=49, mysql_output_write_sql_time=33, ob_sysstat_30200=42, odp_current_session=36, ob_sysstat_30201=42, ob_sysstat_30204=42, ob_sysstat_30205=42, ob_sysstat_30208=42, process_thread_count=41, ob_sysstat_30209=42, ob_sysstat_50040=50, ob_sysstat_40009=49, ob_disk_free_bytes=23, ob_sysstat_40008=49, node_filesystem_avail_bytes=67, ob_sysstat_40007=49, ob_sysstat_40006=49, node_time_seconds=31, ob_unit_config_max_i 2025-11-29 08:44:50.567 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-100,a95473d72be26095,dea6640e5fa9429f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=a95473d72be26095, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:50.567 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,103685b1d9753fa7,5966588aff6bc721] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=103685b1d9753fa7, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:50.567 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-100,a95473d72be26095,dea6640e5fa9429f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=a95473d72be26095, duration=1 ms] 2025-11-29 08:44:50.568 WARN 12717 --- [host-check-clockdiff11,928390af0d622862,38479f22a84d68b6] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.48], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 143 (Exit value: 143) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:50.568 ERROR 12717 --- [host-check-clockdiff11,928390af0d622862,38479f22a84d68b6] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.48] 2025-11-29 08:44:50.568 WARN 12717 --- [host-check-clockdiff11,928390af0d622862,38479f22a84d68b6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=12, ip=10.202.22.48, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.48 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:44:50.575 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-57,103685b1d9753fa7,5966588aff6bc721] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=103685b1d9753fa7, duration=9 ms] 2025-11-29 08:44:50.583 INFO 12717 --- [ocp-schedules-24,c725ad1ada58ba86,c6baa4eb8de9bca7] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-11-29 08:44:50.586 INFO 12717 --- [prometheus-minute-schedule-1,5236657091096834,7d2534f1e2a75205] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:50.646 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:44:50.645663+08:00, lastProcessTime=2025-11-29T08:44:40.510484+08:00 2025-11-29 08:44:50.792 INFO 12717 --- [ocp-schedules-25,324f6c13702f2927,589efde8a5345011] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:44:50.797 INFO 12717 --- [ocp-schedules-25,324f6c13702f2927,589efde8a5345011] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:44:52.285 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:52.292 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:44:50.645663+08:00 2025-11-29 08:44:52.297 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:52.566 INFO 12717 --- [ocp-async-21,f12215971d13fd35,d423b6dc4b65b123] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:53.228 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,71055ff34f6130d1,d6ecdba946aea574] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=71055ff34f6130d1, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:53.228 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-88,71055ff34f6130d1,d6ecdba946aea574] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=71055ff34f6130d1, duration=5 ms] 2025-11-29 08:44:53.407 INFO 12717 --- [prometheus-minute-schedule-1,2878af3dcae65796,75fcd120eec36b55] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:53.441 INFO 12717 --- [prometheus-minute-schedule-1,2878af3dcae65796,75fcd120eec36b55] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:54.266 INFO 12717 --- [ocp-schedules-5,36766f52ba4c3281,9a57fd9315e4dc94] c.o.o.alarm.service.OcpAlarmServiceImpl : fix alarm target, alarms size 48 2025-11-29 08:44:54.428 INFO 12717 --- [ocp-async-35,264fe942f7df97eb,b785b4b70f8b42e7] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:44:54.570 INFO 12717 --- [ocp-async-7,32ac09f807717d44,f417f67e0688503a] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:54.836 INFO 12717 --- [ocp-schedules-26,eb3fb3a87da1b96e,04f51865d83e6215] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:44:54.932 WARN 12717 --- [scanner-11213818520-1,eb3fb3a87da1b96e,e0dabe57c9135bb1] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:44:55.297 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:55.300 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:55.489 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,ff74822516c18b04,b3408f624bde2f06] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=ff74822516c18b04, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:44:55.497 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-92,ff74822516c18b04,b3408f624bde2f06] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=ff74822516c18b04, duration=14 ms] 2025-11-29 08:44:56.409 INFO 12717 --- [prometheus-minute-schedule-1,3459715c2b59b6ad,072a4bfff461155f] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:56.460 INFO 12717 --- [prometheus-minute-schedule-1,3459715c2b59b6ad,072a4bfff461155f] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:56.570 INFO 12717 --- [ocp-async-33,874b3f8fec10b3f2,c1c7b0fa094d5c31] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:56.832 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,4b0d726c03189325,76c1f9fdd09ab8f0] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=4b0d726c03189325, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:56.832 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-79,4b0d726c03189325,76c1f9fdd09ab8f0] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=4b0d726c03189325, duration=5 ms] 2025-11-29 08:44:57.177 INFO 12717 --- [ocp-schedules-7,bb359d1782161a37,4a495704d70b1523] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2025-11-29 08:44:57.177 INFO 12717 --- [ocp-schedules-7,bb359d1782161a37,4a495704d70b1523] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=817192494921281536 2025-11-29 08:44:57.203 INFO 12717 --- [ocp-schedules-7,bb359d1782161a37,4a495704d70b1523] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2025-11-29 08:44:57.203 INFO 12717 --- [ocp-schedules-7,bb359d1782161a37,4a495704d70b1523] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=851074473877958656 2025-11-29 08:44:57.216 INFO 12717 --- [ocp-schedules-7,bb359d1782161a37,4a495704d70b1523] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2025-11-29 08:44:57.216 INFO 12717 --- [ocp-schedules-7,bb359d1782161a37,4a495704d70b1523] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=834880106968510464 2025-11-29 08:44:58.080 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,04573cc097b704c5,e6a8f9d9f3fb0c87] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=04573cc097b704c5, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:44:58.084 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-94,04573cc097b704c5,e6a8f9d9f3fb0c87] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=04573cc097b704c5, duration=5 ms] 2025-11-29 08:44:58.310 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:44:58.315 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:44:58.572 INFO 12717 --- [ocp-async-26,45b346691a0bf1bf,20fc1708a0be15ee] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:44:59.409 INFO 12717 --- [prometheus-minute-schedule-1,f3aece531581d13c,13069d78fbea13a6] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377100, interval=60 2025-11-29 08:44:59.441 INFO 12717 --- [prometheus-minute-schedule-1,f3aece531581d13c,13069d78fbea13a6] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:44:59.709 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-11-29T08:44:59.708057+08:00, lastProcessTime=2025-11-29T08:44:50.645663+08:00 2025-11-29 08:45:00.269 INFO 12717 --- [ocp-schedules-17,124bc70e7e8795fd,7836dad887ad4d05] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=BackupAlarmCheckSchedule, method=run 2025-11-29 08:45:00.274 INFO 12717 --- [ocp-schedules-18,134c2178319657a3,643ee82d8b3b574c] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-11-29 08:45:00.274 INFO 12717 --- [ocp-schedules-14,b3e9205d327a5c9a,005fe5bf052253fa] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-11-29 08:45:00.274 INFO 12717 --- [ocp-schedules-21,e253045b1441eb1d,72be9323fd3dd1dd] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-11-29 08:45:00.280 INFO 12717 --- [ocp-schedules-30,ff12c54295bd0c65,c16d1e466224b20c] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2025-11-29 08:45:00.285 INFO 12717 --- [ocp-schedules-14,b3e9205d327a5c9a,005fe5bf052253fa] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:45:00.419 INFO 12717 --- [ocp-schedules-18,134c2178319657a3,643ee82d8b3b574c] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:45:00.428 INFO 12717 --- [ocp-schedules-30,ff12c54295bd0c65,c16d1e466224b20c] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2025-11-29 08:45:00.433 INFO 12717 --- [ocp-schedules-11,010c775fe2f434db,e37e156bff58a831] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:45:00.438 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:45:00.439 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:45:00.439 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:45:00.439 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:45:00.444 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:45:00.444 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:45:00.444 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:45:00.445 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:45:00.449 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:45:00.450 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:45:00.450 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:45:00.456 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:45:00.508 INFO 12717 --- [ocp-schedules-17,124bc70e7e8795fd,7836dad887ad4d05] c.o.o.b.i.alarm.BackupAlarmService : Backup alarm check schedule. 2025-11-29 08:45:00.527 INFO 12717 --- [ocp-schedules-9,f7d451e06f4ed847,2f424ef940326b7f] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:45:00.528 INFO 12717 --- [host-check-clockdiff1,f7d451e06f4ed847,a7219b5939aecfeb] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.58], timeoutMillis=20000 2025-11-29 08:45:00.529 INFO 12717 --- [host-check-clockdiff2,f7d451e06f4ed847,a0afb5f71dedcaaf] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.50], timeoutMillis=20000 2025-11-29 08:45:00.529 INFO 12717 --- [host-check-clockdiff3,f7d451e06f4ed847,a35a840acd709fd6] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.51], timeoutMillis=20000 2025-11-29 08:45:00.534 INFO 12717 --- [host-check-clockdiff4,f7d451e06f4ed847,39554c7bc0a66068] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.52], timeoutMillis=20000 2025-11-29 08:45:00.541 INFO 12717 --- [host-check-clockdiff5,f7d451e06f4ed847,b18ecf7889a4fd62] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.53], timeoutMillis=20000 2025-11-29 08:45:00.541 INFO 12717 --- [ocp-schedules-29,e1167ba4ac75d612,62190f46a650fec2] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[10.202.22.58, 10.202.22.50, 10.202.22.51, 10.202.22.52, 10.202.22.53, 10.202.22.54, 10.202.22.59, 10.202.22.60, 10.202.22.61, 10.202.22.47, 10.202.22.48, 10.202.22.49] 2025-11-29 08:45:00.542 INFO 12717 --- [host-check-clockdiff7,f7d451e06f4ed847,966f37fd3a5a8ab4] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.59], timeoutMillis=20000 2025-11-29 08:45:00.542 INFO 12717 --- [host-check-clockdiff6,f7d451e06f4ed847,c5a1a817a247d5d7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.54], timeoutMillis=20000 2025-11-29 08:45:00.546 INFO 12717 --- [ocp-schedules-3,370dea6334afe7be,6a16174a6c2ecd6d] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-11-29 08:45:00.547 INFO 12717 --- [host-check-clockdiff8,f7d451e06f4ed847,139c2c0cba9355db] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.60], timeoutMillis=20000 2025-11-29 08:45:00.547 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:45:00.549 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:45:00.552 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:45:00.552 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:45:00.553 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:45:00.554 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:45:00.559 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:45:00.559 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:45:00.560 INFO 12717 --- [host-check-agent-status1,e1167ba4ac75d612,5540d0250bfcc7b4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.202.22.58, originalStatus=ONLINE 2025-11-29 08:45:00.560 INFO 12717 --- [host-check-clockdiff9,f7d451e06f4ed847,9835d832c73fe97c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.61], timeoutMillis=20000 2025-11-29 08:45:00.566 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:45:00.567 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:45:00.567 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:45:00.567 INFO 12717 --- [host-check-clockdiff11,f7d451e06f4ed847,a0805ec4da41addd] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.48], timeoutMillis=20000 2025-11-29 08:45:00.567 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:45:00.568 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:45:00.568 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:45:00.573 INFO 12717 --- [host-check-agent-status3,e1167ba4ac75d612,50859cc057b2d612] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.202.22.51, originalStatus=ONLINE 2025-11-29 08:45:00.574 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:45:00.574 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:45:00.574 INFO 12717 --- [ocp-async-14,fde25c1de0b8814d,17d6fc5f70e41687] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:45:00.574 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:45:00.575 INFO 12717 --- [host-check-clockdiff12,f7d451e06f4ed847,3e62208dffbc0636] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.49], timeoutMillis=20000 2025-11-29 08:45:00.575 INFO 12717 --- [host-check-clockdiff10,f7d451e06f4ed847,ac5a4e71d4d537f4] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.202.22.47], timeoutMillis=20000 2025-11-29 08:45:00.578 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:45:00.588 INFO 12717 --- [ocp-schedules-2,226c6336fc104506,25b5dca1444415e6] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:45:00.592 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:45:00.593 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:45:00.599 INFO 12717 --- [host-check-agent-status4,e1167ba4ac75d612,89b905ed96227d04] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.202.22.52, originalStatus=ONLINE 2025-11-29 08:45:00.600 INFO 12717 --- [host-check-agent-status11,e1167ba4ac75d612,31e6ec8cebe7f3a1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=12, ip=10.202.22.48, originalStatus=ONLINE 2025-11-29 08:45:00.600 INFO 12717 --- [host-check-agent-status12,e1167ba4ac75d612,7e3902b7318dd59a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=13, ip=10.202.22.49, originalStatus=ONLINE 2025-11-29 08:45:00.601 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:45:00.624 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:45:00.630 INFO 12717 --- [host-check-agent-status6,e1167ba4ac75d612,c584633669cc9cad] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.202.22.54, originalStatus=ONLINE 2025-11-29 08:45:00.631 INFO 12717 --- [host-check-agent-status5,e1167ba4ac75d612,dc18862fff1b65f1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.202.22.53, originalStatus=ONLINE 2025-11-29 08:45:00.639 INFO 12717 --- [host-check-agent-status8,e1167ba4ac75d612,800acfce4e47d9b7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=9, ip=10.202.22.60, originalStatus=ONLINE 2025-11-29 08:45:00.646 INFO 12717 --- [host-check-agent-status2,e1167ba4ac75d612,9a7bef02a7244b5c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.202.22.50, originalStatus=ONLINE 2025-11-29 08:45:00.646 INFO 12717 --- [host-check-agent-status7,e1167ba4ac75d612,814d25f95b833414] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=8, ip=10.202.22.59, originalStatus=ONLINE 2025-11-29 08:45:00.647 INFO 12717 --- [host-check-agent-status1,e1167ba4ac75d612,5540d0250bfcc7b4] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:45:00.651 INFO 12717 --- [host-check-agent-status9,e1167ba4ac75d612,cef3cdd074d36f8d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=10, ip=10.202.22.61, originalStatus=ONLINE 2025-11-29 08:45:00.657 INFO 12717 --- [host-check-agent-status10,e1167ba4ac75d612,15cfb8ff1a2dc037] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=11, ip=10.202.22.47, originalStatus=ONLINE 2025-11-29 08:45:00.798 INFO 12717 --- [host-check-agent-status1,e1167ba4ac75d612,5540d0250bfcc7b4] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:45:00.823 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:45:00.825 INFO 12717 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:45:00.844 INFO 12717 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.8 2025-11-29 08:45:00.845 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:45:00.855 INFO 12717 --- [host-check-agent-status3,e1167ba4ac75d612,50859cc057b2d612] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:45:00.861 INFO 12717 --- [host-check-agent-status6,e1167ba4ac75d612,c584633669cc9cad] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:45:00.862 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:45:00.866 INFO 12717 --- [host-check-agent-status9,e1167ba4ac75d612,cef3cdd074d36f8d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:45:00.867 INFO 12717 --- [task-coordinator2,a2d0a58e809f534a,e1a355304a919cb7] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2025-11-29 08:40:00.0, nextRun=2025-11-29 08:45:00.0, currentDate=Sat Nov 29 08:45:00 CST 2025 2025-11-29 08:45:00.867 INFO 12717 --- [host-check-agent-status11,e1167ba4ac75d612,31e6ec8cebe7f3a1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:45:00.867 INFO 12717 --- [task-coordinator2,a2d0a58e809f534a,e1a355304a919cb7] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999977, name=DiagFuncUpdateSchedule, templateName=DiagFuncUpdateSchedule, scheduleRule=0 */5 * * * *, lastRunTime=2025-11-29 08:40:00.0, nextRunTime=2025-11-29 08:45:00.0} 2025-11-29 08:45:00.867 INFO 12717 --- [host-check-agent-status10,e1167ba4ac75d612,15cfb8ff1a2dc037] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:45:00.875 INFO 12717 --- [host-check-agent-status11,e1167ba4ac75d612,31e6ec8cebe7f3a1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:45:00.876 INFO 12717 --- [host-check-agent-status6,e1167ba4ac75d612,c584633669cc9cad] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:45:00.877 INFO 12717 --- [host-check-agent-status9,e1167ba4ac75d612,cef3cdd074d36f8d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:45:00.878 INFO 12717 --- [host-check-agent-status2,e1167ba4ac75d612,9a7bef02a7244b5c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:45:00.881 INFO 12717 --- [host-check-agent-status12,e1167ba4ac75d612,7e3902b7318dd59a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:45:00.881 INFO 12717 --- [host-check-agent-status10,e1167ba4ac75d612,15cfb8ff1a2dc037] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:45:00.882 INFO 12717 --- [host-check-agent-status3,e1167ba4ac75d612,50859cc057b2d612] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:45:00.888 INFO 12717 --- [host-check-agent-status5,e1167ba4ac75d612,dc18862fff1b65f1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:45:00.891 INFO 12717 --- [host-check-agent-status12,e1167ba4ac75d612,7e3902b7318dd59a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:45:00.907 INFO 12717 --- [host-check-agent-status2,e1167ba4ac75d612,9a7bef02a7244b5c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:45:00.907 INFO 12717 --- [host-check-agent-status8,e1167ba4ac75d612,800acfce4e47d9b7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:45:00.907 INFO 12717 --- [host-check-agent-status4,e1167ba4ac75d612,89b905ed96227d04] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:45:00.908 INFO 12717 --- [host-check-agent-status5,e1167ba4ac75d612,dc18862fff1b65f1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:45:00.918 INFO 12717 --- [host-check-agent-status8,e1167ba4ac75d612,800acfce4e47d9b7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:45:00.926 INFO 12717 --- [host-check-agent-status4,e1167ba4ac75d612,89b905ed96227d04] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:45:00.931 INFO 12717 --- [host-check-agent-status7,e1167ba4ac75d612,814d25f95b833414] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:45:00.951 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-11-29 08:45:00.958 INFO 12717 --- [Check status for all clusters2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-11-29 08:45:00.951 INFO 12717 --- [host-check-agent-status7,e1167ba4ac75d612,814d25f95b833414] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:45:00.970 INFO 12717 --- [ocp-schedules-26,7c693f5963c7f984,e27c6badebe03d20] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-11-29 08:45:00.991 INFO 12717 --- [task-coordinator2,a2d0a58e809f534a,e1a355304a919cb7] c.o.o.c.t.e.c.worker.SubtaskGenerator : generate tasks, templateName=DiagFuncUpdateSchedule, nodeCount=1, taskCount=1 2025-11-29 08:45:00.991 INFO 12717 --- [task-coordinator2,a2d0a58e809f534a,e1a355304a919cb7] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name='DiagFuncUpdateSchedule', taskDefinitionId=999977, state=RUNNING, type=SYS_SCHEDULED, clusterId=null, tenantId=null, hostId=null, creator='system', executor='10.202.22.58', argument=Argument{parallelIdx=-1, stringMap={prohibit_rollback=false}, listMap={}}, createTime=null, startTime=2025-11-29T08:45:00.991+08:00, endTime=null, subtasks=[SubtaskInstanceEntity{id=null, name=Update diag function, state=READY, operation=EXECUTE, className=com.oceanbase.ocp.diagnosis.task.DiagFuncUpdateTask, seriesId=0, startTime=null, endTime=null}], template=null, traceId='a2d0a58e809f534a', operation=EXECUTE} 2025-11-29 08:45:01.017 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/time, params:null 2025-11-29 08:45:01.017 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/time, params:null 2025-11-29 08:45:01.023 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.202.22.51, timestamp=2025-11-29T08:45:01.020727937+08:00 2025-11-29 08:45:01.024 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=8, ip=10.202.22.59, timestamp=2025-11-29T08:45:01.023134972+08:00 2025-11-29 08:45:01.032 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-11-29T08:45:01.023+08:00 2025-11-29 08:45:01.032 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=8, lastAvailableTime=2025-11-29T08:45:01.024+08:00 2025-11-29 08:45:01.033 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/time, params:null 2025-11-29 08:45:01.033 INFO 12717 --- [host-check-agent-status1,e1167ba4ac75d612,5540d0250bfcc7b4] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.033 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/time, params:null 2025-11-29 08:45:01.035 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/time, params:null 2025-11-29 08:45:01.035 INFO 12717 --- [ocp-schedules-26,7c693f5963c7f984,e27c6badebe03d20] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-11-29 08:45:01.041 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/time, params:null 2025-11-29 08:45:01.042 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.202.22.54, timestamp=2025-11-29T08:45:01.039668828+08:00 2025-11-29 08:45:01.042 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.s.compute.HostAgentDaoManager : Updated agent 8 last available time 2025-11-29T08:45:01.024+08:00 and status AVAILABLE 2025-11-29 08:45:01.043 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.58:62888/api/v1/time, params:null 2025-11-29 08:45:01.047 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/time, params:null 2025-11-29 08:45:01.047 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2025-11-29T08:45:01.042+08:00 2025-11-29 08:45:01.048 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.202.22.58, timestamp=2025-11-29T08:45:01.047938402+08:00 2025-11-29 08:45:01.048 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-11-29T08:45:01.023+08:00 and status AVAILABLE 2025-11-29 08:45:01.052 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-11-29T08:45:01.048+08:00 2025-11-29 08:45:01.052 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=13, ip=10.202.22.49, timestamp=2025-11-29T08:45:01.048161302+08:00 2025-11-29 08:45:01.053 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=11, ip=10.202.22.47, timestamp=2025-11-29T08:45:01.047376162+08:00 2025-11-29 08:45:01.053 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/time, params:null 2025-11-29 08:45:01.053 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-11-29T08:45:01.042+08:00 and status AVAILABLE 2025-11-29 08:45:01.054 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=13, lastAvailableTime=2025-11-29T08:45:01.052+08:00 2025-11-29 08:45:01.059 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/time, params:null 2025-11-29 08:45:01.059 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.s.compute.HostAgentDaoManager : Updated agent 12 last available time 2025-11-29T08:45:01.052+08:00 and status AVAILABLE 2025-11-29 08:45:01.059 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-11-29T08:45:01.048+08:00 and status AVAILABLE 2025-11-29 08:45:01.065 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=11, lastAvailableTime=2025-11-29T08:45:01.053+08:00 2025-11-29 08:45:01.065 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.202.22.53, timestamp=2025-11-29T08:45:01.060739344+08:00 2025-11-29 08:45:01.067 INFO 12717 --- [host-check-agent-status9,e1167ba4ac75d612,cef3cdd074d36f8d] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.61:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.067 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.s.compute.HostAgentDaoManager : Updated agent 11 last available time 2025-11-29T08:45:01.053+08:00 and status AVAILABLE 2025-11-29 08:45:01.069 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-11-29T08:45:01.065+08:00 2025-11-29 08:45:01.070 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/time, params:null 2025-11-29 08:45:01.075 INFO 12717 --- [host-check-agent-status6,e1167ba4ac75d612,c584633669cc9cad] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.54:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.076 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=12, ip=10.202.22.48, timestamp=2025-11-29T08:45:01.071469284+08:00 2025-11-29 08:45:01.076 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.202.22.50, timestamp=2025-11-29T08:45:01.076050038+08:00 2025-11-29 08:45:01.077 INFO 12717 --- [host-check-agent-status4,e1167ba4ac75d612,89b905ed96227d04] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.080 INFO 12717 --- [Check status for all clusters2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.3.5.1 2025-11-29 08:45:01.081 INFO 12717 --- [host-check-agent-status6,e1167ba4ac75d612,c584633669cc9cad] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.54 2025-11-29 08:45:01.081 INFO 12717 --- [host-check-agent-status6,e1167ba4ac75d612,c584633669cc9cad] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.54 2025-11-29 08:45:01.081 INFO 12717 --- [host-check-agent-status6,e1167ba4ac75d612,c584633669cc9cad] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.54 2025-11-29 08:45:01.081 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-11-29T08:45:01.076+08:00 2025-11-29 08:45:01.081 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-11-29T08:45:01.065+08:00 and status AVAILABLE 2025-11-29 08:45:01.082 INFO 12717 --- [host-check-agent-status10,e1167ba4ac75d612,15cfb8ff1a2dc037] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.47:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.087 INFO 12717 --- [host-check-agent-status12,e1167ba4ac75d612,7e3902b7318dd59a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.49:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.087 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=12, lastAvailableTime=2025-11-29T08:45:01.076+08:00 2025-11-29 08:45:01.087 INFO 12717 --- [host-check-agent-status4,e1167ba4ac75d612,89b905ed96227d04] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.52 2025-11-29 08:45:01.087 INFO 12717 --- [host-check-agent-status4,e1167ba4ac75d612,89b905ed96227d04] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.52 2025-11-29 08:45:01.087 INFO 12717 --- [host-check-agent-status4,e1167ba4ac75d612,89b905ed96227d04] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.52 2025-11-29 08:45:01.091 INFO 12717 --- [host-check-agent-status11,e1167ba4ac75d612,31e6ec8cebe7f3a1] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.48:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.104 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.52:62888/api/v1/time, params:null 2025-11-29 08:45:01.104 INFO 12717 --- [host-check-agent-status12,e1167ba4ac75d612,7e3902b7318dd59a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.49 2025-11-29 08:45:01.104 INFO 12717 --- [host-check-agent-status12,e1167ba4ac75d612,7e3902b7318dd59a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.49 2025-11-29 08:45:01.104 INFO 12717 --- [host-check-agent-status12,e1167ba4ac75d612,7e3902b7318dd59a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.49 2025-11-29 08:45:01.105 INFO 12717 --- [host-check-agent-status10,e1167ba4ac75d612,15cfb8ff1a2dc037] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.47 2025-11-29 08:45:01.105 INFO 12717 --- [host-check-agent-status10,e1167ba4ac75d612,15cfb8ff1a2dc037] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.47 2025-11-29 08:45:01.105 INFO 12717 --- [host-check-agent-status10,e1167ba4ac75d612,15cfb8ff1a2dc037] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.47 2025-11-29 08:45:01.106 INFO 12717 --- [host-check-agent-status3,e1167ba4ac75d612,50859cc057b2d612] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.51:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.106 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2025-11-29T08:45:01.076+08:00 and status AVAILABLE 2025-11-29 08:45:01.116 INFO 12717 --- [host-check-agent-status5,e1167ba4ac75d612,dc18862fff1b65f1] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.53:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.116 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.202.22.52, timestamp=2025-11-29T08:45:01.103541863+08:00 2025-11-29 08:45:01.116 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.s.compute.HostAgentDaoManager : Updated agent 13 last available time 2025-11-29T08:45:01.076+08:00 and status AVAILABLE 2025-11-29 08:45:01.116 INFO 12717 --- [host-check-agent-status3,e1167ba4ac75d612,50859cc057b2d612] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.51 2025-11-29 08:45:01.117 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-11-29T08:45:01.116+08:00 2025-11-29 08:45:01.117 INFO 12717 --- [host-check-agent-status3,e1167ba4ac75d612,50859cc057b2d612] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.51 2025-11-29 08:45:01.121 INFO 12717 --- [host-check-agent-status3,e1167ba4ac75d612,50859cc057b2d612] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.51 2025-11-29 08:45:01.122 INFO 12717 --- [host-check-agent-status5,e1167ba4ac75d612,dc18862fff1b65f1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.53 2025-11-29 08:45:01.122 INFO 12717 --- [host-check-agent-status5,e1167ba4ac75d612,dc18862fff1b65f1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.53 2025-11-29 08:45:01.122 INFO 12717 --- [host-check-agent-status5,e1167ba4ac75d612,dc18862fff1b65f1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.53 2025-11-29 08:45:01.126 INFO 12717 --- [host-check-agent-status2,e1167ba4ac75d612,9a7bef02a7244b5c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.50:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.134 INFO 12717 --- [host-check-agent-status2,e1167ba4ac75d612,9a7bef02a7244b5c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.50 2025-11-29 08:45:01.134 INFO 12717 --- [host-check-agent-status2,e1167ba4ac75d612,9a7bef02a7244b5c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.50 2025-11-29 08:45:01.134 INFO 12717 --- [host-check-agent-status2,e1167ba4ac75d612,9a7bef02a7244b5c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.50 2025-11-29 08:45:01.134 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-11-29T08:45:01.116+08:00 and status AVAILABLE 2025-11-29 08:45:01.135 INFO 12717 --- [host-check-agent-status7,e1167ba4ac75d612,814d25f95b833414] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.59:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.137 INFO 12717 --- [host-check-agent-status8,e1167ba4ac75d612,800acfce4e47d9b7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.202.22.60:62888/api/v1/agent/status, params:null 2025-11-29 08:45:01.153 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,aac37eb738be3aa5,9624428aa6888bca] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.51, traceId=aac37eb738be3aa5, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:45:01.163 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-97,aac37eb738be3aa5,9624428aa6888bca] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=aac37eb738be3aa5, duration=11 ms] 2025-11-29 08:45:01.181 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=10, ip=10.202.22.61, timestamp=2025-11-29T08:45:01.179190342+08:00 2025-11-29 08:45:01.184 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=9, ip=10.202.22.60, timestamp=2025-11-29T08:45:01.173865231+08:00 2025-11-29 08:45:01.184 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=10, lastAvailableTime=2025-11-29T08:45:01.181+08:00 2025-11-29 08:45:01.184 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=9, lastAvailableTime=2025-11-29T08:45:01.184+08:00 2025-11-29 08:45:01.190 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2025-11-29T08:45:01.181+08:00 and status AVAILABLE 2025-11-29 08:45:01.195 INFO 12717 --- [host-check-agent-status9,e1167ba4ac75d612,cef3cdd074d36f8d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.61 2025-11-29 08:45:01.195 INFO 12717 --- [host-check-agent-status9,e1167ba4ac75d612,cef3cdd074d36f8d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.61 2025-11-29 08:45:01.195 INFO 12717 --- [host-check-agent-status9,e1167ba4ac75d612,cef3cdd074d36f8d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.61 2025-11-29 08:45:01.197 INFO 12717 --- [host-check-agent-status1,e1167ba4ac75d612,5540d0250bfcc7b4] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.58 2025-11-29 08:45:01.197 INFO 12717 --- [host-check-agent-status1,e1167ba4ac75d612,5540d0250bfcc7b4] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.58 2025-11-29 08:45:01.197 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.s.compute.HostAgentDaoManager : Updated agent 10 last available time 2025-11-29T08:45:01.184+08:00 and status AVAILABLE 2025-11-29 08:45:01.202 INFO 12717 --- [host-check-agent-status1,e1167ba4ac75d612,5540d0250bfcc7b4] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.58 2025-11-29 08:45:01.203 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-11-29 08:45:01.203 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2025-11-29 08:45:01.203 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-11-29 08:45:01.206 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-11-29 08:45:01.207 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=13 2025-11-29 08:45:01.207 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=8 2025-11-29 08:45:01.207 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=12 2025-11-29 08:45:01.208 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=11 2025-11-29 08:45:01.208 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-11-29 08:45:01.208 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-11-29 08:45:01.213 INFO 12717 --- [host-check-agent-status8,e1167ba4ac75d612,800acfce4e47d9b7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.60 2025-11-29 08:45:01.213 INFO 12717 --- [host-check-agent-status8,e1167ba4ac75d612,800acfce4e47d9b7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.60 2025-11-29 08:45:01.213 INFO 12717 --- [host-check-agent-status8,e1167ba4ac75d612,800acfce4e47d9b7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.60 2025-11-29 08:45:01.218 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,e3e2dca6bc6d82e1,b577ebeece2f7724] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.52, traceId=e3e2dca6bc6d82e1, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:45:01.227 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=9 2025-11-29 08:45:01.229 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=10 2025-11-29 08:45:01.230 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-99,e3e2dca6bc6d82e1,b577ebeece2f7724] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=e3e2dca6bc6d82e1, duration=17 ms] 2025-11-29 08:45:01.264 INFO 12717 --- [task-coordinator2,a2d0a58e809f534a,e1a355304a919cb7] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2025-11-29 08:40:00.0, nextRun=2025-11-29 08:45:00.0, currentDate=Sat Nov 29 08:45:01 CST 2025 2025-11-29 08:45:01.264 INFO 12717 --- [task-coordinator2,a2d0a58e809f534a,e1a355304a919cb7] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999985, name=SqlParseSchedule, templateName=SqlParseSchedule, scheduleRule=0 */5 * * * *, lastRunTime=2025-11-29 08:40:00.0, nextRunTime=2025-11-29 08:45:00.0} 2025-11-29 08:45:01.287 INFO 12717 --- [ocp-schedules-19,16af4f4c501315b1,5dd65bc99ee7c624] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration auto replace status 2025-11-29 08:45:01.291 INFO 12717 --- [host-check-agent-status7,e1167ba4ac75d612,814d25f95b833414] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.59 2025-11-29 08:45:01.291 INFO 12717 --- [host-check-agent-status7,e1167ba4ac75d612,814d25f95b833414] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.59 2025-11-29 08:45:01.291 INFO 12717 --- [host-check-agent-status7,e1167ba4ac75d612,814d25f95b833414] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.59 2025-11-29 08:45:01.293 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-11-29 08:45:01.293 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=13 2025-11-29 08:45:01.294 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=8 2025-11-29 08:45:01.295 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-11-29 08:45:01.296 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-11-29 08:45:01.296 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=11 2025-11-29 08:45:01.297 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-11-29 08:45:01.301 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=9 2025-11-29 08:45:01.302 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-11-29 08:45:01.302 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-11-29 08:45:01.302 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 11 2025-11-29 08:45:01.303 INFO 12717 --- [host-check-agent-status11,e1167ba4ac75d612,31e6ec8cebe7f3a1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.202.22.48 2025-11-29 08:45:01.303 INFO 12717 --- [host-check-agent-status11,e1167ba4ac75d612,31e6ec8cebe7f3a1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.202.22.48 2025-11-29 08:45:01.303 INFO 12717 --- [host-check-agent-status11,e1167ba4ac75d612,31e6ec8cebe7f3a1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.202.22.48 2025-11-29 08:45:01.304 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-11-29 08:45:01.306 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=12 2025-11-29 08:45:01.307 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 9 2025-11-29 08:45:01.308 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 13 2025-11-29 08:45:01.308 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-11-29 08:45:01.308 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 8 2025-11-29 08:45:01.313 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-11-29 08:45:01.318 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-11-29 08:45:01.318 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:45:01.320 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2025-11-29 08:45:01.325 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 12 2025-11-29 08:45:01.327 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:45:01.329 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2025-11-29 08:45:01.331 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=10 2025-11-29 08:45:01.335 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 10 2025-11-29 08:45:01.372 WARN 12717 --- [host-check-clockdiff10,f7d451e06f4ed847,ac5a4e71d4d537f4] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.47], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:01.372 ERROR 12717 --- [host-check-clockdiff10,f7d451e06f4ed847,ac5a4e71d4d537f4] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.47] 2025-11-29 08:45:01.372 WARN 12717 --- [host-check-clockdiff10,f7d451e06f4ed847,ac5a4e71d4d537f4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=11, ip=10.202.22.47, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.47 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:01.424 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.427 INFO 12717 --- [task-coordinator2,a2d0a58e809f534a,e1a355304a919cb7] c.o.o.p.s.internal.SqlParseTaskSchedule : get Clusters for parse:[1, 2] 2025-11-29 08:45:01.430 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.433 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.435 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.451 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.457 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.458 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.462 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.465 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.474 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=16 2025-11-29 08:45:01.480 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.504 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-11-29 08:45:01.504 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-11-29 08:45:01.505 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.505 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=31 2025-11-29 08:45:01.519 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-11-29 08:45:01.526 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=34 2025-11-29 08:45:01.530 INFO 12717 --- [task-coordinator2,a2d0a58e809f534a,e1a355304a919cb7] c.o.o.p.s.internal.SqlParseTaskSchedule : HighRiskSql Parse is ready 2025-11-29 08:45:01.536 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-11-29 08:45:01.614 INFO 12717 --- [host-check-clockdiff1,f7d451e06f4ed847,a7219b5939aecfeb] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764377101 0 0 , error= 2025-11-29 08:45:01.618 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=28 2025-11-29 08:45:01.618 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=25 2025-11-29 08:45:01.618 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-11-29 08:45:01.623 INFO 12717 --- [task-coordinator2,a2d0a58e809f534a,e1a355304a919cb7] c.o.o.c.t.e.c.worker.SubtaskGenerator : generate tasks, templateName=SqlParseTaskSchedule, nodeCount=1, taskCount=1 2025-11-29 08:45:01.623 INFO 12717 --- [task-coordinator2,a2d0a58e809f534a,e1a355304a919cb7] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name='SqlParseTaskSchedule', taskDefinitionId=999985, state=RUNNING, type=SYS_SCHEDULED, clusterId=null, tenantId=null, hostId=null, creator='system', executor='10.202.22.58', argument=Argument{parallelIdx=-1, stringMap={prohibit_rollback=false}, listMap={cluster_ids=[[1,2]]}}, createTime=null, startTime=2025-11-29T08:45:01.623+08:00, endTime=null, subtasks=[SubtaskInstanceEntity{id=null, name=Sql Parse Task, state=READY, operation=EXECUTE, className=com.oceanbase.ocp.perf.sql.internal.task.SqlParseTask, seriesId=0, startTime=null, endTime=null}], template=null, traceId='a2d0a58e809f534a', operation=EXECUTE} 2025-11-29 08:45:01.645 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=16 2025-11-29 08:45:01.646 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-11-29 08:45:01.646 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=31 2025-11-29 08:45:01.652 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-11-29 08:45:01.652 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-11-29 08:45:01.667 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=34 2025-11-29 08:45:01.684 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=28 2025-11-29 08:45:01.699 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=25 2025-11-29 08:45:01.704 INFO 12717 --- [subtask-coordinator2,a2d0a58e809f534a,326f1d2ee95c5f46] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 89000224, operation:EXECUTE, state: RUNNING 2025-11-29 08:45:01.705 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=22 2025-11-29 08:45:01.716 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-11-29 08:45:01.739 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=22 2025-11-29 08:45:01.789 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=37 2025-11-29 08:45:01.790 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-11-29 08:45:01.835 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-11-29 08:45:01.867 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=37 2025-11-29 08:45:01.933 INFO 12717 --- [alarm-task-2,d90e23be01739268,b7f1fa0eccfb6511] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-11-29T08:44:59.708057+08:00 2025-11-29 08:45:01.973 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:01.984 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:01.986 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:01.986 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:01.986 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:01.986 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:01.989 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:01.989 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:01.997 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:02.012 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:02.007 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:02.025 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-11-29 08:45:02.031 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,c0df5e18c78901f6,f92ecf7e202b849a] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.53, traceId=c0df5e18c78901f6, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:45:02.034 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-59,c0df5e18c78901f6,f92ecf7e202b849a] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=c0df5e18c78901f6, duration=10 ms] 2025-11-29 08:45:02.040 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.ocp.service.iam.user.UserService : user 200 login with organization 10000000 2025-11-29 08:45:02.042 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=17 2025-11-29 08:45:02.046 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-11-29 08:45:02.046 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=89000224, context=Context{parallelIdx=-1, stringMap={task_instance_id=89000231, task_operation=execute, latest_execution_start_time=2025-11-29T08:45:01.618+08:00, sub_task_instance_name=Update diag function, sub_task_instance_id=89000224}, listMap={}}, executor=10.202.22.58 2025-11-29 08:45:02.046 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=29 2025-11-29 08:45:02.051 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=23 2025-11-29 08:45:02.051 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-11-29 08:45:02.052 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=32 2025-11-29 08:45:02.052 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=26 2025-11-29 08:45:02.060 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-11-29 08:45:02.079 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=35 2025-11-29 08:45:02.084 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-11-29 08:45:02.086 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=17 2025-11-29 08:45:02.087 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=29 2025-11-29 08:45:02.092 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-11-29 08:45:02.096 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=26 2025-11-29 08:45:02.104 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=32 2025-11-29 08:45:02.105 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=23 2025-11-29 08:45:02.106 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-11-29 08:45:02.119 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-11-29 08:45:02.130 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-11-29 08:45:02.148 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=35 2025-11-29 08:45:02.162 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:ocpserver 2025-11-29 08:45:02.180 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-11-29 08:45:02.234 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-11-29 08:45:02.263 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=38 2025-11-29 08:45:02.280 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.o.d.o.DiskUsageTimeRemainingOperator : do least Square for ocpserver 2025-11-29 08:45:02.280 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.ocp.diagnosis.model.NeuralNetwork : W:[73.4614373448897], b:[-1.1805152331171814E11] 2025-11-29 08:45:02.327 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=38 2025-11-29 08:45:02.338 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.60:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.338 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:ocpserver 2025-11-29 08:45:02.344 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.52:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.350 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.53:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.359 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.59:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.362 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.47:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.368 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.49:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.368 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.58:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.368 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.51:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.375 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.54:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.377 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.50:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.388 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.61:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.401 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-11-29 08:45:02.406 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=30 2025-11-29 08:45:02.407 INFO 12717 --- [prometheus-minute-schedule-1,af5f782c9e33afc5,2d5d335ac60984a1] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377160, interval=60 2025-11-29 08:45:02.417 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=33 2025-11-29 08:45:02.413 WARN 12717 --- [host-check-clockdiff2,f7d451e06f4ed847,a0afb5f71dedcaaf] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.50], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:02.417 ERROR 12717 --- [host-check-clockdiff2,f7d451e06f4ed847,a0afb5f71dedcaaf] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.50] 2025-11-29 08:45:02.417 WARN 12717 --- [host-check-clockdiff2,f7d451e06f4ed847,a0afb5f71dedcaaf] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.202.22.50, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.50 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:02.423 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=24 2025-11-29 08:45:02.423 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-11-29 08:45:02.430 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-11-29 08:45:02.431 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=18 2025-11-29 08:45:02.436 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=36 2025-11-29 08:45:02.443 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.202.22.48:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-11-29 08:45:02.457 INFO 12717 --- [prometheus-minute-schedule-1,af5f782c9e33afc5,2d5d335ac60984a1] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:45:02.464 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-11-29 08:45:02.465 WARN 12717 --- [host-check-clockdiff4,f7d451e06f4ed847,39554c7bc0a66068] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.52], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:02.471 ERROR 12717 --- [host-check-clockdiff4,f7d451e06f4ed847,39554c7bc0a66068] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.52] 2025-11-29 08:45:02.471 WARN 12717 --- [host-check-clockdiff4,f7d451e06f4ed847,39554c7bc0a66068] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.202.22.52, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.52 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:02.478 INFO 12717 --- [host-check-host-status8,010c775fe2f434db,9142e371984fc331] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=30 2025-11-29 08:45:02.483 INFO 12717 --- [host-check-host-status4,010c775fe2f434db,cb5805f5cad270ca] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-11-29 08:45:02.495 INFO 12717 --- [host-check-host-status10,010c775fe2f434db,f6a87c7c91f29dbe] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=33 2025-11-29 08:45:02.506 INFO 12717 --- [host-check-host-status5,010c775fe2f434db,598c75404ff1ab48] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-11-29 08:45:02.508 INFO 12717 --- [host-check-host-status3,010c775fe2f434db,4d9e4177ca991284] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-11-29 08:45:02.513 INFO 12717 --- [host-check-host-status7,010c775fe2f434db,81a133e98aab0a70] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=24 2025-11-29 08:45:02.513 INFO 12717 --- [host-check-host-status12,010c775fe2f434db,3c10af3f5e3ff375] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=36 2025-11-29 08:45:02.529 INFO 12717 --- [host-check-host-status2,010c775fe2f434db,cb7879f63132af7c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=18 2025-11-29 08:45:02.534 INFO 12717 --- [host-check-host-status6,010c775fe2f434db,72c2cab44879e720] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-11-29 08:45:02.535 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-11-29 08:45:02.576 INFO 12717 --- [ocp-async-8,e5b5992a7d7a1749,577426b219dd807d] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:45:02.576 INFO 12717 --- [host-check-host-status1,010c775fe2f434db,3b62355d6359cf8c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-11-29 08:45:02.658 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=27 2025-11-29 08:45:02.726 INFO 12717 --- [host-check-host-status9,010c775fe2f434db,fa158eb338b15a16] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=27 2025-11-29 08:45:02.778 WARN 12717 --- [host-check-clockdiff5,f7d451e06f4ed847,b18ecf7889a4fd62] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.53], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:02.778 ERROR 12717 --- [host-check-clockdiff5,f7d451e06f4ed847,b18ecf7889a4fd62] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.53] 2025-11-29 08:45:02.778 WARN 12717 --- [host-check-clockdiff5,f7d451e06f4ed847,b18ecf7889a4fd62] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.202.22.53, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.53 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:02.783 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:ocpserver 2025-11-29 08:45:02.811 WARN 12717 --- [host-check-clockdiff12,f7d451e06f4ed847,3e62208dffbc0636] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.49], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:02.815 ERROR 12717 --- [host-check-clockdiff12,f7d451e06f4ed847,3e62208dffbc0636] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.49] 2025-11-29 08:45:02.816 WARN 12717 --- [host-check-clockdiff12,f7d451e06f4ed847,3e62208dffbc0636] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=13, ip=10.202.22.49, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.49 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:02.949 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=39 2025-11-29 08:45:02.987 INFO 12717 --- [host-check-host-status11,010c775fe2f434db,99bca12e02932111] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=39 2025-11-29 08:45:02.991 INFO 12717 --- [subtask-coordinator2,a2d0a58e809f534a,28ba40e96ae113b1] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 89000225, operation:EXECUTE, state: RUNNING 2025-11-29 08:45:03.114 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.ocp.service.iam.user.UserService : user 200 login with organization 10000000 2025-11-29 08:45:03.158 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=89000225, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=89000232, task_operation=execute, latest_execution_start_time=2025-11-29T08:45:02.981+08:00, sub_task_instance_name=Sql Parse Task, sub_task_instance_id=89000225}, listMap={cluster_ids=[[1,2]]}}, executor=10.202.22.58 2025-11-29 08:45:03.159 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.p.sql.internal.task.SqlParseTask : parse sql for cluster:1 2025-11-29 08:45:03.167 WARN 12717 --- [host-check-clockdiff3,f7d451e06f4ed847,a35a840acd709fd6] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.51], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:03.167 ERROR 12717 --- [host-check-clockdiff3,f7d451e06f4ed847,a35a840acd709fd6] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.51] 2025-11-29 08:45:03.167 WARN 12717 --- [host-check-clockdiff3,f7d451e06f4ed847,a35a840acd709fd6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.202.22.51, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.51 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:03.286 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1743215593,ocpserver,1002,1764374400000000,2000,30000000 2025-11-29 08:45:03.478 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : parse 0 sqls for tenant:ObTenantEntity{id=2, name=ocp_meta, obTenantId=1002, status=NORMAL} 2025-11-29 08:45:03.570 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1743215593,ocpserver,1004,1764374400000000,2000,30000000 2025-11-29 08:45:03.574 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : parse 0 sqls for tenant:ObTenantEntity{id=3, name=ocp_monitor, obTenantId=1004, status=NORMAL} 2025-11-29 08:45:03.574 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.p.sql.internal.task.SqlParseTask : parse sql for cluster:2 2025-11-29 08:45:03.791 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1743215594,DB,1010,1764374400000000,2000,30000000 2025-11-29 08:45:03.805 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : parse 0 sqls for tenant:ObTenantEntity{id=9, name=lhp6796, obTenantId=1010, status=NORMAL} 2025-11-29 08:45:03.829 WARN 12717 --- [host-check-clockdiff6,f7d451e06f4ed847,c5a1a817a247d5d7] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.54], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:03.829 ERROR 12717 --- [host-check-clockdiff6,f7d451e06f4ed847,c5a1a817a247d5d7] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.54] 2025-11-29 08:45:03.829 WARN 12717 --- [host-check-clockdiff6,f7d451e06f4ed847,c5a1a817a247d5d7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.202.22.54, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.54 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:03.899 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1743215594,DB,1018,1764374400000000,2000,30000000 2025-11-29 08:45:03.909 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : parse 6 sqls for tenant:ObTenantEntity{id=13, name=nextcloud, obTenantId=1018, status=NORMAL} 2025-11-29 08:45:03.938 INFO 12717 --- [print-time-series-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Async compute queue max size = 0, current size = 0. 2025-11-29 08:45:03.981 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:ocpserver 2025-11-29 08:45:04.016 INFO 12717 --- [ocp-schedules-29,60d36da09ae715eb,f021e7cd07cf6f03] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog instances, size=0 2025-11-29 08:45:04.327 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:45:04.334 INFO 12717 --- [pool-13-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:45:04.431 INFO 12717 --- [ocp-async-28,41eda8f89f3bb70b,86756dd76c0644b7] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-11-29 08:45:04.579 INFO 12717 --- [ocp-async-8,e9d576277bcd50ff,a4b30637331b0662] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:45:04.629 INFO 12717 --- [ocp-schedules-20,35addd488c04ee58,89c538f5358a177d] c.o.o.a.service.OcpAlarmSearchService : alarmTargetStat [] 2025-11-29 08:45:04.639 INFO 12717 --- [ocp-schedules-20,35addd488c04ee58,89c538f5358a177d] c.o.o.a.service.OcpAlarmSearchService : alarmTargetStat [] 2025-11-29 08:45:04.666 WARN 12717 --- [host-check-clockdiff8,f7d451e06f4ed847,139c2c0cba9355db] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.60], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:04.666 ERROR 12717 --- [host-check-clockdiff8,f7d451e06f4ed847,139c2c0cba9355db] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.60] 2025-11-29 08:45:04.666 WARN 12717 --- [host-check-clockdiff8,f7d451e06f4ed847,139c2c0cba9355db] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=9, ip=10.202.22.60, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.60 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:05.097 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:ocpserver 2025-11-29 08:45:05.133 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,cf1daeb88d9e3921,4381a8688a85faba] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.52, traceId=cf1daeb88d9e3921, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:45:05.133 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-103,cf1daeb88d9e3921,4381a8688a85faba] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.52, traceId=cf1daeb88d9e3921, duration=5 ms] 2025-11-29 08:45:05.189 INFO 12717 --- [ocp-schedules-3,7227a3c241c85b66,b760b599e65f6580] c.o.o.s.c.scheduled.ClusterSchedules : Collect all cluster compaction info. 2025-11-29 08:45:05.200 INFO 12717 --- [ocp-schedules-3,7227a3c241c85b66,b760b599e65f6580] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1, 2] 2025-11-29 08:45:05.200 INFO 12717 --- [Collect compaction for all clusters1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-11-29 08:45:05.200 INFO 12717 --- [Collect compaction for all clusters2,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-11-29 08:45:05.306 INFO 12717 --- [ocp-schedules-17,8b8b7bf757d20331,1013b287c8977b92] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-11-29 08:45:05.356 INFO 12717 --- [alarm-detect-13,8b8b7bf757d20331,aa744d39f3784afc] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008372, data is empty. 2025-11-29 08:45:05.361 INFO 12717 --- [alarm-detect-6,8b8b7bf757d20331,65136bb9ce75c608] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008372, data is empty. 2025-11-29 08:45:05.362 INFO 12717 --- [alarm-detect-7,8b8b7bf757d20331,1f0af1f42819753f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008372, data is empty. 2025-11-29 08:45:05.393 INFO 12717 --- [alarm-detect-7,8b8b7bf757d20331,1f0af1f42819753f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 83008372, data is empty. 2025-11-29 08:45:05.397 INFO 12717 --- [alarm-detect-13,8b8b7bf757d20331,aa744d39f3784afc] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 83008372, data is empty. 2025-11-29 08:45:05.408 INFO 12717 --- [prometheus-minute-schedule-1,2048838fa2484c63,c027b40099994190] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377160, interval=60 2025-11-29 08:45:05.419 INFO 12717 --- [alarm-detect-6,8b8b7bf757d20331,65136bb9ce75c608] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 83008372, data is empty. 2025-11-29 08:45:05.427 WARN 12717 --- [scanner-11213818785-1,8b8b7bf757d20331,7525c3de8fc3aedb] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 24 2025-11-29 08:45:05.446 INFO 12717 --- [prometheus-minute-schedule-1,2048838fa2484c63,c027b40099994190] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=36, collectTaskCount=36 2025-11-29 08:45:05.608 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-77,6db3627d1f3be1ba,9824fa292b4ee9e9] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=OBP&VersionOnly=true, client=10.202.22.51, traceId=6db3627d1f3be1ba, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,OBP,true,] 2025-11-29 08:45:05.608 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-77,6db3627d1f3be1ba,9824fa292b4ee9e9] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.51, traceId=6db3627d1f3be1ba, duration=12 ms] 2025-11-29 08:45:05.644 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1743215594,DB,1014,1764374400000000,2000,30000000 2025-11-29 08:45:05.651 INFO 12717 --- [alarm-detect-13,8b8b7bf757d20331,cb8581912993ad9a] c.o.o.alarm.core.detect.MetricDetector : alarmType ob_tenant_cpu_usage_over_threshold detect time range 1764377065 ~ 1764377095 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricSnapshots={ob_tenant_cpu_percent=[SeriesMetricValues(metric=Metric{name='ob_tenant_cpu_percent', labels={app=OB, host=10.202.22.58, host_name=ocp-22.58, ob_cluster_id=1743215593, ob_cluster_name=ocpserver, ob_tenant_id=1004, obzone=OCP, svr_ip=10.202.22.58, tenant_name=ocp_monitor}}, data=[ValueData(timestamp=1764377085, value=234.233)])]}, metricEvaluateResults={ob_tenant_cpu_percent > 95=[ExpressionEvaluateResult(matchValue=95.0, metric=ob_tenant_cpu_percent, match=true, value=234.233, values={1764377085=234.233}, valueCount=1, labels={app=OB, host=10.202.22.58, host_name=ocp-22.58, ob_cluster_id=1743215593, ob_cluster_name=ocpserver, ob_tenant_id=1004, obzone=OCP, svr_ip=10.202.22.58, tenant_name=ocp_monitor})]}, matched=true, isContainValidValue=null, message=null, level=2) 2025-11-29 08:45:05.651 INFO 12717 --- [alarm-detect-13,8b8b7bf757d20331,cb8581912993ad9a] c.o.o.alarm.core.detect.MetricDetector : alarmType ob_tenant_cpu_usage_over_threshold detect time range 1764377065 ~ 1764377095 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricSnapshots={ob_tenant_cpu_percent=[SeriesMetricValues(metric=Metric{name='ob_tenant_cpu_percent', labels={app=OB, host=10.202.22.58, host_name=ocp-22.58, ob_cluster_id=1743215593, ob_cluster_name=ocpserver, ob_tenant_id=1002, obzone=OCP, svr_ip=10.202.22.58, tenant_name=ocp_meta}}, data=[ValueData(timestamp=1764377085, value=319.733)])]}, metricEvaluateResults={ob_tenant_cpu_percent > 95=[ExpressionEvaluateResult(matchValue=95.0, metric=ob_tenant_cpu_percent, match=true, value=319.733, values={1764377085=319.733}, valueCount=1, labels={app=OB, host=10.202.22.58, host_name=ocp-22.58, ob_cluster_id=1743215593, ob_cluster_name=ocpserver, ob_tenant_id=1002, obzone=OCP, svr_ip=10.202.22.58, tenant_name=ocp_meta})]}, matched=true, isContainValidValue=null, message=null, level=2) 2025-11-29 08:45:05.655 INFO 12717 --- [alarm-detect-13,8b8b7bf757d20331,cb8581912993ad9a] c.o.o.a.c.detect.AlarmRuleDetectState : firstDetect, alarmType=ob_tenant_cpu_usage_over_threshold, target=alarm_template_id=1:ob_cluster=ocpserver-1743215593:tenant_name=ocp_meta:host=10.202.22.58-2, currentState=Pending 2025-11-29 08:45:05.667 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : parse 0 sqls for tenant:ObTenantEntity{id=11, name=oms, obTenantId=1014, status=NORMAL} 2025-11-29 08:45:05.743 INFO 12717 --- [alarm-detect-20,8b8b7bf757d20331,d2a014a00eeef2db] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008372, data is empty. 2025-11-29 08:45:05.787 INFO 12717 --- [alarm-detect-20,8b8b7bf757d20331,d2a014a00eeef2db] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 83008372, data is empty. 2025-11-29 08:45:05.849 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1743215594,DB,1020,1764374400000000,2000,30000000 2025-11-29 08:45:05.874 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : parse 3 sqls for tenant:ObTenantEntity{id=14, name=pipeline, obTenantId=1020, status=NORMAL} 2025-11-29 08:45:06.089 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,4a6455639fbb60b8,7adfecaa5aea8e46] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObIDCRegionInfo&User_ID=alibaba&UID=test&ObRegion=DB, client=10.202.22.53, traceId=4a6455639fbb60b8, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObIDCRegionInfo(String,Long,String,Long,Integer), args=DB,0,] 2025-11-29 08:45:06.118 INFO 12717 --- [http-nio-0.0.0.0-8080-exec-32,4a6455639fbb60b8,7adfecaa5aea8e46] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.202.22.53, traceId=4a6455639fbb60b8, duration=38 ms] 2025-11-29 08:45:06.558 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:ocpserver 2025-11-29 08:45:06.579 INFO 12717 --- [ocp-async-6,1cae5b1b24ecbb23,dbef4fe9e62fd4fb] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:45:07.336 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:45:07.337 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-11-29 08:45:07.459 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1743215594,DB,1024,1764374400000000,2000,30000000 2025-11-29 08:45:07.472 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : parse 0 sqls for tenant:ObTenantEntity{id=16, name=wecom_bot, obTenantId=1024, status=NORMAL} 2025-11-29 08:45:07.627 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1743215594,DB,1022,1764374400000000,2000,30000000 2025-11-29 08:45:07.638 INFO 12717 --- [pool-17-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : Starting SQL_PERF_DIAGNOSER, Interval{start=2025-11-29T08:39:05.178, end=2025-11-29T08:44:07.282} 2025-11-29 08:45:07.642 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.perf.sql.internal.SqlParseService : parse 0 sqls for tenant:ObTenantEntity{id=15, name=zto, obTenantId=1022, status=NORMAL} 2025-11-29 08:45:07.673 INFO 12717 --- [pool-17-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : SQL_PERF_DIAGNOSER : for [Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Pair(left=Tenant(id=3, name=ocp_monitor, clusterName=ocpserver, obClusterId=1743215593, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-29T10:44:04+08:00, primaryZone=RANDOM, zoneList=OCP, locality=FULL{1}@OCP, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.8, description=null, obTenantId=1004, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Pair(left=Tenant(id=14, name=pipeline, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-04-29T17:10:07+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1020, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTe 2025-11-29 08:45:07.820 INFO 12717 --- [pool-17-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : ROW_LOCK_CONTENTION_HIGH found out 0 diagnose targets Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:39:05.178, end=2025-11-29T08:44:07.282} 2025-11-29 08:45:07.862 INFO 12717 --- [pool-17-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : PERFORMANCE_DEGRADATION found out 3 diagnose targets Pair(left=Tenant(id=9, name=lhp6796, clusterName=DB, obClusterId=1743215594, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-03-31T15:30:45+08:00, primaryZone=RANDOM, zoneList=Hangzhou;zone_1;zone_3, locality=FULL{1}@Hangzhou, FULL{1}@zone_1, FULL{1}@zone_3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.1, description=null, obTenantId=1010, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=HTAP), right=null), Interval{start=2025-11-29T08:39:05.178, end=2025-11-29T08:44:07.282} 2025-11-29 08:45:08.002 INFO 12717 --- [subtask-executor26,a2d0a58e809f534a,17df45d6a5b11dfb] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:ocpserver 2025-11-29 08:45:08.003 INFO 12717 --- [subtask-executor31,a2d0a58e809f534a,8cb598f3a52cc49b] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 89000225, operation:EXECUTE, state: SUCCESSFUL 2025-11-29 08:45:08.410 INFO 12717 --- [prometheus-minute-schedule-1,f08d0d16443f27a8,d1efccc3305af94b] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1764377160, interval=60 2025-11-29 08:45:08.566 WARN 12717 --- [host-check-clockdiff11,f7d451e06f4ed847,a0805ec4da41addd] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.202.22.48], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:497) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:08.566 ERROR 12717 --- [host-check-clockdiff11,f7d451e06f4ed847,a0805ec4da41addd] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.202.22.48] 2025-11-29 08:45:08.567 WARN 12717 --- [host-check-clockdiff11,f7d451e06f4ed847,a0805ec4da41addd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=12, ip=10.202.22.48, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.202.22.48 at sun.reflect.GeneratedConstructorAccessor512.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:499) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:486) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-11-29 08:45:08.583 INFO 12717 --- [ocp-async-33,09e496c64a643667,823f0f71ee2da816] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:45:09.949 INFO 12717 --- [host-check-clockdiff7,f7d451e06f4ed847,966f37fd3a5a8ab4] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1764377108 0 -1 , error= 2025-11-29 08:45:10.300 INFO 12717 --- [ocp-schedules-14,b3e9205d327a5c9a,005fe5bf052253fa] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=2, successCount=2 2025-11-29 08:45:10.337 INFO 12717 --- [pool-13-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.202.22.58:8080 refreshes policies... 2025-11-29 08:45:10.609 INFO 12717 --- [ocp-async-21,6020df7f252a2edd,6e25a12a60e86003] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-11-29 08:45:10.622 WARN 12717 --- [ocp-async-21,6020df7f252a2edd,6e25a12a60e86003] c.o.o.s.common.DistributedLockAspect : Acquire lock fail, another request is processing, key=AlarmEventProcessForSelfcure 2025-11-29 08:45:10.650 INFO 12717 --- [ocp-schedules-10,7cec25b63ea6eaa1,ef0d4d528b3a6e9b] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster zone cgroup value. 2025-11-29 08:45:10.658 INFO 12717 --- [ocp-schedules-4,1591843c921611ba,398f9a3e808eccf8] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-11-29 08:45:11.454 INFO 12717 --- [ocp-schedules-18,134c2178319657a3,643ee82d8b3b574c] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=2, successCount=2 2025-11-29 08:45:12.715 INFO 12717 --- [ocp-async-21,850d966693320ea6,477dfae40c1c7b31] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents