2025-09-02 13:35:28.921 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:28.921 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:29.091 INFO 13484 --- [ocp-async-1,afa0e0c482a80ed9,bddf1b11bd59ff19] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:35:29.289 INFO 13484 --- [ocp-schedules-21,95d28a37acc1278e,e292df928ab448d1] c.o.o.s.c.scheduled.TenantSchedules : Sync all tenant info schedule. 2025-09-02 13:35:29.291 INFO 13484 --- [ocp-schedules-21,95d28a37acc1278e,e292df928ab448d1] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:35:29.335 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-09-02 13:35:29.336 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.339 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) ocp_query */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-09-02 13:35:29.353 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.354 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-09-02 13:35:29.381 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.381 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-09-02 13:35:29.408 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.408 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1] 2025-09-02 13:35:29.412 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.412 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2025-09-02 13:35:29.438 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.438 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2025-09-02 13:35:29.463 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.464 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1002] 2025-09-02 13:35:29.478 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.479 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2025-09-02 13:35:29.481 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.482 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-09-02 13:35:29.484 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.485 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-09-02 13:35:29.488 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.489 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2025-09-02 13:35:29.491 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.492 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-09-02 13:35:29.495 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.496 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-09-02 13:35:29.498 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2025-09-02 13:35:29.500 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=172.19.62.162 2025-09-02 13:35:29.527 INFO 13484 --- [Sync all tenant information1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:35:29.529 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:29.529 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE name = ?, args: [ob_tcp_invited_nodes] 2025-09-02 13:35:29.541 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,172.19.62.162, userWhitelist=% 2025-09-02 13:35:29.541 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2025-09-02 13:35:29.541 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,172.19.62.162, userWhitelist=% 2025-09-02 13:35:29.541 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2025-09-02 13:35:29.544 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2025-09-02 13:35:29.544 INFO 13484 --- [Sync all tenant information1,,] .o.s.t.b.t.SyncAllTenantInfoShardingTask : [OCP606] sync tenants success, clusterId=1, clusterName=myoceanbase, obClusterId=1755130302 2025-09-02 13:35:29.544 INFO 13484 --- [ocp-schedules-21,95d28a37acc1278e,e292df928ab448d1] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all tenant information executed completed, allCount=1, successCount=1 2025-09-02 13:35:30.419 INFO 13484 --- [ocp-async-4,c23453a073981d79,1ce9446cab078399] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:30.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,bf83334e19638d2c,d060e7d6366de1c3] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=bf83334e19638d2c, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:35:30.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,bf83334e19638d2c,d060e7d6366de1c3] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=bf83334e19638d2c, duration=5 ms] 2025-09-02 13:35:30.923 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,2bf5b719a854d6a2,5baccc6f3fc318a7] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=2bf5b719a854d6a2, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:35:30.925 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,0d4eedc1186170cf,e9c6582685511be5] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=0d4eedc1186170cf, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:35:30.929 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,0d4eedc1186170cf,e9c6582685511be5] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=0d4eedc1186170cf, duration=6 ms] 2025-09-02 13:35:30.931 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,2d290cff5e08faa5,ca9238d04e3536d5] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=2d290cff5e08faa5, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:35:30.931 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,2d290cff5e08faa5,ca9238d04e3536d5] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=2d290cff5e08faa5, duration=9 ms] 2025-09-02 13:35:30.932 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,2bf5b719a854d6a2,5baccc6f3fc318a7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=2bf5b719a854d6a2, duration=15 ms] 2025-09-02 13:35:31.323 INFO 13484 --- [prometheus-minute-schedule-1,bbb23a4fffd31594,a0a3a88c8c1abf29] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791360, interval=60 2025-09-02 13:35:31.328 INFO 13484 --- [prometheus-minute-schedule-1,bbb23a4fffd31594,a0a3a88c8c1abf29] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:35:31.921 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:31.922 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:32.419 INFO 13484 --- [ocp-async-7,947cd9ab01a4f5bc,4caf85ea06303a4c] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:32.637 INFO 13484 --- [ocp-schedules-8,8b9122162c34b3bb,5be13ce72c922a65] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:35:32.639 INFO 13484 --- [ocp-schedules-8,8b9122162c34b3bb,5be13ce72c922a65] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:35:34.323 INFO 13484 --- [prometheus-minute-schedule-1,c36f62878a377148,ca0d0a2095cbd12e] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791360, interval=60 2025-09-02 13:35:34.327 INFO 13484 --- [prometheus-minute-schedule-1,c36f62878a377148,ca0d0a2095cbd12e] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:35:34.419 INFO 13484 --- [ocp-async-1,b21b079a79f51e51,6ad41ea10ee84ac2] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:34.914 INFO 13484 --- [ocp-schedules-17,79aa06cacb7e60ce,30459996f014693f] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-09-02 13:35:34.923 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:34.923 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:35.512 INFO 13484 --- [ocp-schedules-28,379bd1dcd37c45e6,a720e6313bb1e8b5] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:35:35.519 WARN 13484 --- [scanner-12207591-1,379bd1dcd37c45e6,7ebdf61bb0971971] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:35:35.549 INFO 13484 --- [alarm-detect-20,379bd1dcd37c45e6,fb5e7de7ad814773] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 3016164, data is empty. 2025-09-02 13:35:35.555 INFO 13484 --- [alarm-detect-8,379bd1dcd37c45e6,97d3b17beeb8839b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 3016164, data is empty. 2025-09-02 13:35:35.558 INFO 13484 --- [alarm-detect-17,379bd1dcd37c45e6,ec41c8d3e310fca4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 3016164, data is empty. 2025-09-02 13:35:35.597 INFO 13484 --- [alarm-detect-7,379bd1dcd37c45e6,6b7893658e89378c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 3016164, data is empty. 2025-09-02 13:35:35.906 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,,] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /error?null, client=114.221.202.171, traceId=, method=ModelAndView com.oceanbase.ocp.server.common.controller.web.CustomErrorController.handleError(HttpServletRequest), args=] 2025-09-02 13:35:35.906 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-1,,] c.o.o.s.c.c.web.CustomErrorController : Requested URL not found: [url=/obproxy, method=114.221.202.171, client=GET, traceId=] 2025-09-02 13:35:35.906 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,,] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /error client=114.221.202.171, traceId=, duration=0 ms] 2025-09-02 13:35:36.133 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.s.common.scheduled.OcpSchedules : Refresh OCP properties. 2025-09-02 13:35:36.133 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.ocp.core.property.PropertyService : Refresh config properties 2025-09-02 13:35:36.137 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : new: [] 2025-09-02 13:35:36.137 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : init spring 2025-09-02 13:35:36.137 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : have already run initialize. 2025-09-02 13:35:36.137 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : starting 2025-09-02 13:35:36.141 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : new: [] 2025-09-02 13:35:36.141 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : init spring 2025-09-02 13:35:36.141 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : have already run initialize. 2025-09-02 13:35:36.141 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : starting 2025-09-02 13:35:36.170 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : environmentPrepared 2025-09-02 13:35:36.175 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : contextPrepared 2025-09-02 13:35:36.175 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : set internal properties as the first PropertySource 2025-09-02 13:35:36.176 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : register bean ProgressHandler 2025-09-02 13:35:36.176 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : set internal property ocp.version to 4.3.6 2025-09-02 13:35:36.176 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : ocp.version in spring: 4.3.6 2025-09-02 13:35:36.182 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : contextLoaded 2025-09-02 13:35:36.419 INFO 13484 --- [ocp-async-8,e9aefce41f36a765,692d508b73f55c25] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:36.546 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,dc560ed80b874da3,424b6f945d3e7486] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /favicon.ico?null, client=114.221.202.171, traceId=dc560ed80b874da3, method=void com.oceanbase.ocp.server.common.controller.web.WebController.dummyFavicon(), args=] 2025-09-02 13:35:36.546 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,dc560ed80b874da3,424b6f945d3e7486] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /favicon.ico client=114.221.202.171, traceId=dc560ed80b874da3, duration=2 ms] 2025-09-02 13:35:36.552 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.DataSourceInterceptor : dataSource dataSource already initialized. skipped. class: org.springframework.jdbc.datasource.DriverManagerDataSource 2025-09-02 13:35:36.790 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,1dbf7e5ae19ce4a1,49992ccb182f4e1d] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/authInfo?null, client=114.221.202.171, traceId=1dbf7e5ae19ce4a1, method=SuccessResponse com.oceanbase.ocp.server.common.controller.InfoController.getAuthInfo(), args=] 2025-09-02 13:35:36.793 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,1dbf7e5ae19ce4a1,49992ccb182f4e1d] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/authInfo client=114.221.202.171, traceId=1dbf7e5ae19ce4a1, duration=8 ms] 2025-09-02 13:35:36.986 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:35:36.985970+08:00, lastProcessTime=2025-09-02T13:35:26.984498+08:00 2025-09-02 13:35:36.993 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : ready. time taken: PT0.851948932S 2025-09-02 13:35:36.993 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : environmentPrepared 2025-09-02 13:35:37.076 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] o.s.c.c.s.e.NativeEnvironmentRepository : Adding property source: Config resource 'class path resource [config/ocp.yaml]' via location 'classpath:/config/' 2025-09-02 13:35:37.076 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] o.s.c.c.s.e.NativeEnvironmentRepository : Adding property source: Config resource 'class path resource [application.properties]' via location 'classpath:/' 2025-09-02 13:35:37.099 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] b.c.PropertySourceBootstrapConfiguration : Located property source: [BootstrapPropertySource {name='bootstrapProperties-classpath:/config/ocp.yaml'}, BootstrapPropertySource {name='bootstrapProperties-classpath:/application.properties'}, BootstrapPropertySource {name='bootstrapProperties-ocp-default'}] 2025-09-02 13:35:37.152 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : contextPrepared 2025-09-02 13:35:37.152 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : set internal properties as the first PropertySource 2025-09-02 13:35:37.152 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : register bean ProgressHandler 2025-09-02 13:35:37.152 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : set internal property ocp.version to 4.3.6 2025-09-02 13:35:37.152 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : ocp.version in spring: 4.3.6 2025-09-02 13:35:37.153 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] o.s.boot.SpringApplication : The following 1 profile is active: "composite" 2025-09-02 13:35:37.164 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : contextLoaded 2025-09-02 13:35:37.166 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:35:36.985970+08:00 2025-09-02 13:35:37.309 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] o.s.boot.SpringApplication : Started application in 1.171 seconds (JVM running for 84386.624) 2025-09-02 13:35:37.309 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.o.b.spring.BootstrapRunListener : ready. time taken: PT1.172146257S 2025-09-02 13:35:37.316 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,,] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /error?null, client=114.221.202.171, traceId=, method=ModelAndView com.oceanbase.ocp.server.common.controller.web.CustomErrorController.handleError(HttpServletRequest), args=] 2025-09-02 13:35:37.316 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-5,,] c.o.o.s.c.c.web.CustomErrorController : Requested URL not found: [url=/obproxy, method=114.221.202.171, client=GET, traceId=] 2025-09-02 13:35:37.316 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,,] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /error client=114.221.202.171, traceId=, duration=0 ms] 2025-09-02 13:35:37.323 INFO 13484 --- [prometheus-minute-schedule-1,ed36cbdebad5e83b,801a8b3cfabaf28e] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791360, interval=60 2025-09-02 13:35:37.327 INFO 13484 --- [prometheus-minute-schedule-1,ed36cbdebad5e83b,801a8b3cfabaf28e] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:35:37.615 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,bf28d0b5d3bbe450,98d4dddbff46ebd7] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /favicon.ico?null, client=114.221.202.171, traceId=bf28d0b5d3bbe450, method=void com.oceanbase.ocp.server.common.controller.web.WebController.dummyFavicon(), args=] 2025-09-02 13:35:37.615 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,bf28d0b5d3bbe450,98d4dddbff46ebd7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /favicon.ico client=114.221.202.171, traceId=bf28d0b5d3bbe450, duration=4 ms] 2025-09-02 13:35:37.664 INFO 13484 --- [ocp-schedules-10,0e3407118b814380,d0c53d14e9012ae8] c.o.ocp.core.property.PropertyService : Refresh config properties done 2025-09-02 13:35:37.780 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,5edf7680f16004f0,529ef40cf86b7a0f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/authInfo?null, client=114.221.202.171, traceId=5edf7680f16004f0, method=SuccessResponse com.oceanbase.ocp.server.common.controller.InfoController.getAuthInfo(), args=] 2025-09-02 13:35:37.785 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,5edf7680f16004f0,529ef40cf86b7a0f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/authInfo client=114.221.202.171, traceId=5edf7680f16004f0, duration=26 ms] 2025-09-02 13:35:37.924 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:37.925 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:38.216 INFO 13484 --- [ocp-schedules-1,cd2ea048612037cd,a53085762f208a59] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:35:38.279 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,,] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /error?null, client=114.221.202.171, traceId=, method=ModelAndView com.oceanbase.ocp.server.common.controller.web.CustomErrorController.handleError(HttpServletRequest), args=] 2025-09-02 13:35:38.279 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-2,,] c.o.o.s.c.c.web.CustomErrorController : Requested URL not found: [url=/obproxy, method=114.221.202.171, client=GET, traceId=] 2025-09-02 13:35:38.279 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,,] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /error client=114.221.202.171, traceId=, duration=0 ms] 2025-09-02 13:35:38.420 INFO 13484 --- [ocp-async-10,175eef691060baf1,b619838774f12f73] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:38.549 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,272f2233ec38bbb6,084e415e100ce273] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /favicon.ico?null, client=114.221.202.171, traceId=272f2233ec38bbb6, method=void com.oceanbase.ocp.server.common.controller.web.WebController.dummyFavicon(), args=] 2025-09-02 13:35:38.549 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,272f2233ec38bbb6,084e415e100ce273] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /favicon.ico client=114.221.202.171, traceId=272f2233ec38bbb6, duration=2 ms] 2025-09-02 13:35:38.673 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,27bdf263144a773f,8fd0a67ebd6d5ba0] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/authInfo?null, client=114.221.202.171, traceId=27bdf263144a773f, method=SuccessResponse com.oceanbase.ocp.server.common.controller.InfoController.getAuthInfo(), args=] 2025-09-02 13:35:38.675 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,27bdf263144a773f,8fd0a67ebd6d5ba0] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/authInfo client=114.221.202.171, traceId=27bdf263144a773f, duration=6 ms] 2025-09-02 13:35:39.092 INFO 13484 --- [ocp-async-4,b0f553397b69d41d,b0ab35412981a81d] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:35:39.189 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,9c380a3f58bd6714,28077f89e2df2f0f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /?null, client=114.221.202.171, traceId=9c380a3f58bd6714, method=ModelAndView com.oceanbase.ocp.server.common.controller.web.WebController.index(), args=] 2025-09-02 13:35:39.190 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,9c380a3f58bd6714,28077f89e2df2f0f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET / client=114.221.202.171, traceId=9c380a3f58bd6714, duration=4 ms] 2025-09-02 13:35:39.438 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,e9bb5cede208c57d,7ef115075be95973] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /favicon.ico?null, client=114.221.202.171, traceId=e9bb5cede208c57d, method=void com.oceanbase.ocp.server.common.controller.web.WebController.dummyFavicon(), args=] 2025-09-02 13:35:39.438 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,e9bb5cede208c57d,7ef115075be95973] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /favicon.ico client=114.221.202.171, traceId=e9bb5cede208c57d, duration=3 ms] 2025-09-02 13:35:39.549 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,41b77104d0fe4785,d1e186f2fb985e15] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/authInfo?null, client=114.221.202.171, traceId=41b77104d0fe4785, method=SuccessResponse com.oceanbase.ocp.server.common.controller.InfoController.getAuthInfo(), args=] 2025-09-02 13:35:39.549 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,c6f1a355b5b9cb6e,7e4d25876962d6c3] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/config/systemInfo?null, client=114.221.202.171, traceId=c6f1a355b5b9cb6e, method=SuccessResponse com.oceanbase.ocp.server.common.controller.PropertyController.getSystemInfo(), args=] 2025-09-02 13:35:39.549 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,c6f1a355b5b9cb6e,7e4d25876962d6c3] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/config/systemInfo client=114.221.202.171, traceId=c6f1a355b5b9cb6e, duration=3 ms] 2025-09-02 13:35:39.549 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,ed8f31ac1e60a9e8,65d29c01fcc6b84b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/profiles/me?null, client=114.221.202.171, traceId=ed8f31ac1e60a9e8, method=SuccessResponse com.oceanbase.ocp.server.common.controller.ProfileController.userInfo(), args=] 2025-09-02 13:35:39.551 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,41b77104d0fe4785,d1e186f2fb985e15] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/authInfo client=114.221.202.171, traceId=41b77104d0fe4785, duration=5 ms] 2025-09-02 13:35:39.554 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,ed8f31ac1e60a9e8,65d29c01fcc6b84b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/profiles/me client=114.221.202.171, traceId=ed8f31ac1e60a9e8, duration=8 ms] 2025-09-02 13:35:39.634 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,271b1da73e343cbf,3ee8f2b77f63b29d] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/telemetry?null, client=114.221.202.171, traceId=271b1da73e343cbf, method=SuccessResponse com.oceanbase.ocp.controller.TelemetryController.get(), args=] 2025-09-02 13:35:39.645 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,271b1da73e343cbf,3ee8f2b77f63b29d] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/telemetry client=114.221.202.171, traceId=271b1da73e343cbf, duration=15 ms] 2025-09-02 13:35:39.701 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,51afe669a46ae257,a564326f7d31c303] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/alarm/alarms/actived?null, client=114.221.202.171, traceId=51afe669a46ae257, method=PaginatedResponse com.oceanbase.ocp.server.common.controller.AlarmController.fetchActivedAlarms(String,String,List,String,List,List,List,OffsetDateTime,OffsetDateTime,String,Pageable), args=Page request [number: 0, size 10, sort: id: DESC],] 2025-09-02 13:35:39.713 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,,] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /error?null, client=114.221.202.171, traceId=, method=ModelAndView com.oceanbase.ocp.server.common.controller.web.CustomErrorController.handleError(HttpServletRequest), args=] 2025-09-02 13:35:39.713 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-8,,] c.o.o.s.c.c.web.CustomErrorController : Requested URL not found: [url=/api/v2/mc/info, method=114.221.202.171, client=GET, traceId=] 2025-09-02 13:35:39.713 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,,] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /error client=114.221.202.171, traceId=, duration=0 ms] 2025-09-02 13:35:39.882 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,51afe669a46ae257,a564326f7d31c303] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/alarm/alarms/actived client=114.221.202.171, traceId=51afe669a46ae257, duration=186 ms] 2025-09-02 13:35:40.005 INFO 13484 --- [ocp-schedules-25,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=2 2025-09-02 13:35:40.005 INFO 13484 --- [ocp-schedules-30,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=2 2025-09-02 13:35:40.022 INFO 13484 --- [Check obproxy2,2e45610cf3eb00d6,aa8823043c85beb8] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:35:40.023 INFO 13484 --- [Check obproxy1,d4bb7a36bd1e7007,9d5a50f45f32e002] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:35:40.025 INFO 13484 --- [Collect obproxy2,d2e5f6d4221c57a4,51931cfb4367ee4e] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:35:40.026 INFO 13484 --- [Collect obproxy1,489bfc44203d0eb0,c638ec92ecda9886] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:35:40.028 INFO 13484 --- [Check obproxy2,2e45610cf3eb00d6,aa8823043c85beb8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:35:40.030 INFO 13484 --- [Collect obproxy1,489bfc44203d0eb0,c638ec92ecda9886] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:35:40.033 INFO 13484 --- [Check obproxy1,d4bb7a36bd1e7007,9d5a50f45f32e002] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:35:40.035 INFO 13484 --- [Collect obproxy2,d2e5f6d4221c57a4,51931cfb4367ee4e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:35:40.053 INFO 13484 --- [Check obproxy2,2e45610cf3eb00d6,aa8823043c85beb8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:35:40.056 INFO 13484 --- [Check obproxy2,2e45610cf3eb00d6,aa8823043c85beb8] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-09-02 13:35:40.056 INFO 13484 --- [Check obproxy2,2e45610cf3eb00d6,aa8823043c85beb8] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 5, need to sync to meta: false 2025-09-02 13:35:40.062 INFO 13484 --- [Check obproxy1,d4bb7a36bd1e7007,9d5a50f45f32e002] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:35:40.062 INFO 13484 --- [Collect obproxy1,489bfc44203d0eb0,c638ec92ecda9886] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:35:40.066 INFO 13484 --- [Collect obproxy1,489bfc44203d0eb0,c638ec92ecda9886] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:35:40.066 INFO 13484 --- [Check obproxy1,d4bb7a36bd1e7007,9d5a50f45f32e002] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-09-02 13:35:40.066 INFO 13484 --- [Check obproxy1,d4bb7a36bd1e7007,9d5a50f45f32e002] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 6, need to sync to meta: false 2025-09-02 13:35:40.067 INFO 13484 --- [Collect obproxy2,d2e5f6d4221c57a4,51931cfb4367ee4e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:35:40.069 INFO 13484 --- [Collect obproxy1,489bfc44203d0eb0,c638ec92ecda9886] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:35:40.070 INFO 13484 --- [Collect obproxy2,d2e5f6d4221c57a4,51931cfb4367ee4e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:35:40.073 INFO 13484 --- [Collect obproxy2,d2e5f6d4221c57a4,51931cfb4367ee4e] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:35:40.192 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,7356d01eaa510460,7a20db12639b3402] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/ob/clusters/stats/topAvgCompactionDuration?limit=5&inDays=7, client=114.221.202.171, traceId=7356d01eaa510460, method=IterableResponse com.oceanbase.ocp.server.common.controller.ObClusterStatsController.getClustersTopAvgCompactionDuration(Integer,Integer), args=7,5,] 2025-09-02 13:35:40.193 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,7709fe33922f30fc,f6261edc0a21ee9c] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=7709fe33922f30fc, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:35:40.194 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,4d706726641d05ce,0c008de23b5fbc8c] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/ob/clusters/tenants/topAvgCompactionDuration?limit=5&inDays=7, client=114.221.202.171, traceId=4d706726641d05ce, method=IterableResponse com.oceanbase.ocp.server.common.controller.ObTenantCompactionController.topAvgCompactionDuration(Integer,Integer), args=7,5,] 2025-09-02 13:35:40.194 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,452fb2078ef536e5,9d536bcd46c3c96b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=452fb2078ef536e5, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:35:40.194 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,7709fe33922f30fc,f6261edc0a21ee9c] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=7709fe33922f30fc, duration=4 ms] 2025-09-02 13:35:40.196 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,7356d01eaa510460,7a20db12639b3402] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/ob/clusters/stats/topAvgCompactionDuration client=114.221.202.171, traceId=7356d01eaa510460, duration=9 ms] 2025-09-02 13:35:40.196 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,452fb2078ef536e5,9d536bcd46c3c96b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=452fb2078ef536e5, duration=6 ms] 2025-09-02 13:35:40.209 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,c5e409d2c3012e57,12aab6ffefe70251] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=c5e409d2c3012e57, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:35:40.210 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,c5e409d2c3012e57,12aab6ffefe70251] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=c5e409d2c3012e57, duration=9 ms] 2025-09-02 13:35:40.212 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,05d9d63234e62d3f,2b4bdc4b0c0a1ab2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=05d9d63234e62d3f, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:35:40.212 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,05d9d63234e62d3f,2b4bdc4b0c0a1ab2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=05d9d63234e62d3f, duration=9 ms] 2025-09-02 13:35:40.214 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,4d706726641d05ce,0c008de23b5fbc8c] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/ob/clusters/tenants/topAvgCompactionDuration client=114.221.202.171, traceId=4d706726641d05ce, duration=24 ms] 2025-09-02 13:35:40.231 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,0cc216b9c4f0ecc5,e82a08f433e143ce] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/monitor/metricGroups?scope=OVERVIEW_TENANT&type=TOP, client=114.221.202.171, traceId=0cc216b9c4f0ecc5, method=PaginatedResponse com.oceanbase.ocp.server.common.controller.MonitorController.listMetricClasses(Pageable,MetricType,MetricScope,VersionTarget,Long), args=Page request [number: 0, size 5000, sort: id: ASC],TOP,OVERVIEW_TENANT,OBCLUSTER,] 2025-09-02 13:35:40.234 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,8366adcf869ec313,a04237706ddf966f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/monitor/metricGroups?scope=OVERVIEW_ODP&type=TOP, client=114.221.202.171, traceId=8366adcf869ec313, method=PaginatedResponse com.oceanbase.ocp.server.common.controller.MonitorController.listMetricClasses(Pageable,MetricType,MetricScope,VersionTarget,Long), args=Page request [number: 0, size 5000, sort: id: ASC],TOP,OVERVIEW_ODP,OBCLUSTER,] 2025-09-02 13:35:40.238 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,8366adcf869ec313,a04237706ddf966f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/monitor/metricGroups client=114.221.202.171, traceId=8366adcf869ec313, duration=9 ms] 2025-09-02 13:35:40.238 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,0cc216b9c4f0ecc5,e82a08f433e143ce] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/monitor/metricGroups client=114.221.202.171, traceId=0cc216b9c4f0ecc5, duration=11 ms] 2025-09-02 13:35:40.248 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,09a74378bf49ab02,4c3e92604b00573e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/monitor/metricGroups?scope=OVERVIEW_HOST&type=TOP, client=114.221.202.171, traceId=09a74378bf49ab02, method=PaginatedResponse com.oceanbase.ocp.server.common.controller.MonitorController.listMetricClasses(Pageable,MetricType,MetricScope,VersionTarget,Long), args=Page request [number: 0, size 5000, sort: id: ASC],TOP,OVERVIEW_HOST,OBCLUSTER,] 2025-09-02 13:35:40.254 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,f2a75de600dafe10,e2abe73ec0357423] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/cluster/resources?metrics=ob_disk_used_percentage&labels=app%3AOB&groupBy=app%2Cob_cluster_name%2Cob_cluster_id, client=114.221.202.171, traceId=f2a75de600dafe10, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewClusterResources(String,String,String,Integer), args=ob_disk_used_percentage,app,ob_cluster_name,ob_cluster_id,app:OB,5,] 2025-09-02 13:35:40.254 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,65e35769b0c0bc47,19daefd6a65e1941] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/cluster/resources?metrics=ob_memory_assigned_percentage&labels=app%3AOB&groupBy=app%2Cob_cluster_name%2Cob_cluster_id, client=114.221.202.171, traceId=65e35769b0c0bc47, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewClusterResources(String,String,String,Integer), args=ob_memory_assigned_percentage,app,ob_cluster_name,ob_cluster_id,app:OB,5,] 2025-09-02 13:35:40.256 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,09a74378bf49ab02,4c3e92604b00573e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/monitor/metricGroups client=114.221.202.171, traceId=09a74378bf49ab02, duration=14 ms] 2025-09-02 13:35:40.258 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,1035821efdb46f8d,330856e41db7a90d] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/cluster/resources?metrics=ob_cpu_assigned_percentage&labels=app%3AOB&groupBy=app%2Cob_cluster_name%2Cob_cluster_id, client=114.221.202.171, traceId=1035821efdb46f8d, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewClusterResources(String,String,String,Integer), args=ob_cpu_assigned_percentage,app,ob_cluster_name,ob_cluster_id,app:OB,5,] 2025-09-02 13:35:40.266 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-6,65e35769b0c0bc47,19daefd6a65e1941] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async get empty result, metric = ob_memory_assigned_percentage 2025-09-02 13:35:40.266 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-6,65e35769b0c0bc47,19daefd6a65e1941] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async result range is away from start time, fall back to full query. metric = ob_memory_assigned_percentage, minTimestamp = 0, minTimeBuffer = 0, startTime = 1756790760 2025-09-02 13:35:40.266 WARN 13484 --- [scanner-12207746-1,65e35769b0c0bc47,64392a2e4551d61b] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = ob_server_resource_memory_assigned_bytes, seriesId size = 3 2025-09-02 13:35:40.266 WARN 13484 --- [scanner-12207747-1,65e35769b0c0bc47,b658d6d227f54c14] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = ob_server_resource_memory_bytes, seriesId size = 3 2025-09-02 13:35:40.267 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,65e35769b0c0bc47,19daefd6a65e1941] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/cluster/resources client=114.221.202.171, traceId=65e35769b0c0bc47, duration=15 ms] 2025-09-02 13:35:40.268 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-2,1035821efdb46f8d,330856e41db7a90d] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async get empty result, metric = ob_cpu_assigned_percentage 2025-09-02 13:35:40.268 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-2,1035821efdb46f8d,330856e41db7a90d] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async result range is away from start time, fall back to full query. metric = ob_cpu_assigned_percentage, minTimestamp = 0, minTimeBuffer = 0, startTime = 1756790760 2025-09-02 13:35:40.268 WARN 13484 --- [scanner-12207748-1,1035821efdb46f8d,ab387015b2a01ec1] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = ob_server_resource_cpu_assigned, seriesId size = 3 2025-09-02 13:35:40.270 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,307afe87c5ffb325,f2a95b9b064da178] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/tenant/resources?metrics=ob_memory_percent&labels=app%3AOB&groupBy=app%2Cob_cluster_name%2Cob_cluster_id%2Ctenant_name, client=114.221.202.171, traceId=307afe87c5ffb325, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewTenantResources(String,String,String,Integer), args=ob_memory_percent,app,ob_cluster_name,ob_cluster_id,tenant_name,app:OB,5,] 2025-09-02 13:35:40.270 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-10,f2a75de600dafe10,e2abe73ec0357423] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async get empty result, metric = ob_disk_used_percentage 2025-09-02 13:35:40.270 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-10,f2a75de600dafe10,e2abe73ec0357423] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async result range is away from start time, fall back to full query. metric = ob_disk_used_percentage, minTimestamp = 0, minTimeBuffer = 0, startTime = 1756790760 2025-09-02 13:35:40.277 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,9f33a15a5517d99e,31b87cf36de66da2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/tenant/resources?metrics=ob_cpu_percent&labels=app%3AOB&groupBy=app%2Cob_cluster_name%2Cob_cluster_id%2Ctenant_name, client=114.221.202.171, traceId=9f33a15a5517d99e, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewTenantResources(String,String,String,Integer), args=ob_cpu_percent,app,ob_cluster_name,ob_cluster_id,tenant_name,app:OB,5,] 2025-09-02 13:35:40.277 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-3,307afe87c5ffb325,f2a95b9b064da178] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async get empty result, metric = ob_memory_percent 2025-09-02 13:35:40.277 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-3,307afe87c5ffb325,f2a95b9b064da178] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async result range is away from start time, fall back to full query. metric = ob_memory_percent, minTimestamp = 0, minTimeBuffer = 0, startTime = 1756790760 2025-09-02 13:35:40.280 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-5,9f33a15a5517d99e,31b87cf36de66da2] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async cannot find recordId. metric = ob_cpu_percent, labels = , groupBy = app,ob_cluster_name,ob_cluster_id,tenant_name, interval = 60 2025-09-02 13:35:40.286 WARN 13484 --- [scanner-12207749-1,1035821efdb46f8d,f1a3ba51c1a0f77f] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = ob_server_resource_cpu, seriesId size = 3 2025-09-02 13:35:40.287 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,1035821efdb46f8d,330856e41db7a90d] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/cluster/resources client=114.221.202.171, traceId=1035821efdb46f8d, duration=42 ms] 2025-09-02 13:35:40.299 WARN 13484 --- [scanner-12207750-1,f2a75de600dafe10,112725d9530952b2] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = ob_disk_free_bytes, seriesId size = 3 2025-09-02 13:35:40.299 WARN 13484 --- [scanner-12207754-1,f2a75de600dafe10,a75369f25cf9ff7b] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = ob_disk_total_bytes, seriesId size = 3 2025-09-02 13:35:40.300 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,f2a75de600dafe10,e2abe73ec0357423] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/cluster/resources client=114.221.202.171, traceId=f2a75de600dafe10, duration=52 ms] 2025-09-02 13:35:40.303 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,9f33a15a5517d99e,31b87cf36de66da2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/tenant/resources client=114.221.202.171, traceId=9f33a15a5517d99e, duration=30 ms] 2025-09-02 13:35:40.304 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,900a1019fec44f6a,6c10ce6212f864b0] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/tenant/resources?metrics=ob_tenant_disk_usage_v4&labels=app%3AOB&groupBy=app%2Cob_cluster_name%2Cob_cluster_id%2Ctenant_name, client=114.221.202.171, traceId=900a1019fec44f6a, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewTenantResources(String,String,String,Integer), args=ob_tenant_disk_usage_v4,app,ob_cluster_name,ob_cluster_id,tenant_name,app:OB,5,] 2025-09-02 13:35:40.304 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,307afe87c5ffb325,f2a95b9b064da178] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/tenant/resources client=114.221.202.171, traceId=307afe87c5ffb325, duration=38 ms] 2025-09-02 13:35:40.311 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-9,900a1019fec44f6a,6c10ce6212f864b0] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async get empty result, metric = ob_tenant_disk_usage_v4 2025-09-02 13:35:40.311 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-9,900a1019fec44f6a,6c10ce6212f864b0] c.o.o.m.s.MonitorAsyncComputeServiceImpl : Scan data async result range is away from start time, fall back to full query. metric = ob_tenant_disk_usage_v4, minTimestamp = 0, minTimeBuffer = 0, startTime = 1756790760 2025-09-02 13:35:40.312 WARN 13484 --- [scanner-12207756-1,900a1019fec44f6a,3b57b44c1cb99962] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = ob_tenant_server_required_size, seriesId size = 6 2025-09-02 13:35:40.312 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,900a1019fec44f6a,6c10ce6212f864b0] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/tenant/resources client=114.221.202.171, traceId=900a1019fec44f6a, duration=19 ms] 2025-09-02 13:35:40.322 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,3ea60585e7691c1a,72aada5664713bc1] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/ob/clusters?foldStandby=false, client=114.221.202.171, traceId=3ea60585e7691c1a, method=PaginatedResponse com.oceanbase.ocp.server.common.controller.ObClusterController.listObClusters(String,List,boolean,Boolean,WithRemote,List,Pageable), args=false,NONE,Page request [number: 0, size 5000, sort: name: ASC,obClusterId: ASC],] 2025-09-02 13:35:40.323 INFO 13484 --- [prometheus-minute-schedule-1,7a5a0427126bdbbb,3b8c6672d12a1f8a] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791360, interval=60 2025-09-02 13:35:40.327 INFO 13484 --- [prometheus-minute-schedule-1,7a5a0427126bdbbb,3b8c6672d12a1f8a] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:35:40.333 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,3ea60585e7691c1a,72aada5664713bc1] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/ob/clusters client=114.221.202.171, traceId=3ea60585e7691c1a, duration=19 ms] 2025-09-02 13:35:40.420 INFO 13484 --- [ocp-async-5,40abf2f3749f5143,f2b0da4e6d186425] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:40.765 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,04e0c341c05e170a,535f4c4157ae20cf] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/profiles/me?null, client=114.221.202.171, traceId=04e0c341c05e170a, method=SuccessResponse com.oceanbase.ocp.server.common.controller.ProfileController.userInfo(), args=] 2025-09-02 13:35:40.771 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,04e0c341c05e170a,535f4c4157ae20cf] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/profiles/me client=114.221.202.171, traceId=04e0c341c05e170a, duration=10 ms] 2025-09-02 13:35:40.925 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:40.926 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:41.323 INFO 13484 --- [prometheus-cache-monitor1,652b8051f14111d2,f09cdd4cbd63eb7f] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":10713,"minuteWriteQueue":0,"minuteReadCacheSize":4820,"secondWriteCacheSize":2678,"seriesIdKeySize":5749,"secondWriteQueue":0,"minuteWriteCacheSize":4806} 2025-09-02 13:35:41.324 INFO 13484 --- [prometheus-cache-monitor1,652b8051f14111d2,f09cdd4cbd63eb7f] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{ob_sysstat_40012=15, ob_server_resource_memory_assigned_bytes=10, ob_sysstat_40011=15, ob_sysstat_40010=15, ob_unit_num=10, ob_database_disk_required_size=29, go_memstats_mspan_sys_bytes=16, ob_sysstat_30213=15, go_memstats_stack_sys_bytes=16, ob_sysstat_30212=15, ob_sysstat_30217=15, ob_sysstat_30216=15, ob_tenant_request_queue_total_size=16, jvm_gc_memory_allocated_bytes_total=2, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=10, ob_sysstat_40019=15, ob_sysstat_40018=15, processor_process_metrics_total=20, jvm_gc_max_data_size_bytes=2, ob_sysstat_40027=15, ob_sysstat_40026=15, ob_sysstat_40025=15, go_memstats_frees_total=16, host_available=15, ob_sysstat_30220=15, ob_sysstat_30222=15, ob_sysstat_30221=15, mysql_output_write_sql_failed_count=19, ob_database_disk_data_size=28, ob_tenant_memstore_freeze_cnt=12, ob_tenant_unit_min_memory=21, ob_sysstat_40028=15, ob_sysstat_60087=15, ob_memstore_total_bytes=16, jvm_threads_states_threads=8, process_resident_memory_bytes=16, ob_cache_size_bytes=33, ob_tenant_unit_max_memory=21, input_collect_metrics_total=24, ob_sysstat_50039=15, ob_sysstat_60083=15, ob_sysstat_50038=15, node_disk_io_time_weighted_seconds_total=27, process_open_fds=16, ob_sysstat_50037=15, process_memory_rss_bytes=29, ob_sysstat_50030=15, ob_server_resource_cpu_assigned_percent=10, sql_audit_input_collect_lost_total=12, go_memstats_heap_inuse_bytes=16, ob_sysstat_40118=15, ob_sysstat_40117=15, node_memory_memfree_bytes=20, ob_sysstat_40116=15, ob_sysstat_40005=15, process_exists=39, ob_sysstat_40004=15, ob_sysstat_40003=15, ob_sysstat_40002=15, go_gc_duration_seconds=21, ob_server_resource_disk_bytes=10, ob_sysstat_40001=15, ob_sysstat_40000=15, mysql_output_write_sql_time=19, ob_sysstat_30200=15, odp_current_session=17, ob_sysstat_30201=15, ob_sysstat_30204=15, ob_sysstat_30205=15, ob_sysstat_30208=15, process_thread_count=29, ob_sysstat_30209=15, ob_sysstat_50040=15, ob_sysstat_40009=15, ob_disk_free_bytes=10, ob_sysstat_40008=15, node_filesystem_avail_bytes=49, 2025-09-02 13:35:41.528 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,f3232e6bfe1abf18,b7f90e4ead06384d] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/sensitiveDataKey?null, client=114.221.202.171, traceId=f3232e6bfe1abf18, method=SuccessResponse com.oceanbase.ocp.server.common.controller.InfoController.getSensitiveDataPublicKey(), args=] 2025-09-02 13:35:41.528 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,0957a4c90395394f,3c1a3585c6227b0a] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/info?null, client=114.221.202.171, traceId=0957a4c90395394f, method=OcpBuildInfo com.oceanbase.ocp.server.common.controller.InfoController.info(), args=] 2025-09-02 13:35:41.528 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,0957a4c90395394f,3c1a3585c6227b0a] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/info client=114.221.202.171, traceId=0957a4c90395394f, duration=4 ms] 2025-09-02 13:35:41.528 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,f3232e6bfe1abf18,b7f90e4ead06384d] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/sensitiveDataKey client=114.221.202.171, traceId=f3232e6bfe1abf18, duration=3 ms] 2025-09-02 13:35:41.529 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,e350c75b1d1da7ee,5c8c715ab3e7f259] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/time?null, client=114.221.202.171, traceId=e350c75b1d1da7ee, method=OffsetDateTime com.oceanbase.ocp.server.common.controller.InfoController.time(), args=] 2025-09-02 13:35:41.529 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,e350c75b1d1da7ee,5c8c715ab3e7f259] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/time client=114.221.202.171, traceId=e350c75b1d1da7ee, duration=5 ms] 2025-09-02 13:35:41.545 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,,] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /error?null, client=114.221.202.171, traceId=, method=ModelAndView com.oceanbase.ocp.server.common.controller.web.CustomErrorController.handleError(HttpServletRequest), args=] 2025-09-02 13:35:41.545 WARN 13484 --- [http-nio-0.0.0.0-8080-exec-4,,] c.o.o.s.c.c.web.CustomErrorController : Requested URL not found: [url=/api/v2/maz/state, method=114.221.202.171, client=GET, traceId=] 2025-09-02 13:35:41.545 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,,] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /error client=114.221.202.171, traceId=, duration=0 ms] 2025-09-02 13:35:42.420 INFO 13484 --- [ocp-async-2,098e8838e80b2aa4,586b82cdfd640685] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:42.647 INFO 13484 --- [ocp-schedules-13,562608229ceb143e,c1a3a1ab00c700af] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:35:42.650 INFO 13484 --- [ocp-schedules-13,562608229ceb143e,c1a3a1ab00c700af] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:35:43.323 INFO 13484 --- [prometheus-minute-schedule-1,5ffd9d734ae5be37,46bab25cfa3cd036] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791360, interval=60 2025-09-02 13:35:43.327 INFO 13484 --- [prometheus-minute-schedule-1,5ffd9d734ae5be37,46bab25cfa3cd036] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:35:43.719 INFO 13484 --- [ocp-schedules-15,3a185efbea208722,65a4c64818850683] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-09-02 13:35:43.727 INFO 13484 --- [ocp-schedules-15,3a185efbea208722,65a4c64818850683] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:35:43.728 INFO 13484 --- [ocp-schedules-15,3a185efbea208722,65a4c64818850683] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:35:43.926 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:43.928 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:44.420 INFO 13484 --- [ocp-async-9,9e28f53b8f0993d5,9d34bb5672b72679] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:44.925 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,8c4eb1e8ed8d04dd,83fb487568e8d128] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/config/systemInfo?null, client=114.221.202.171, traceId=8c4eb1e8ed8d04dd, method=SuccessResponse com.oceanbase.ocp.server.common.controller.PropertyController.getSystemInfo(), args=] 2025-09-02 13:35:44.925 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,8c4eb1e8ed8d04dd,83fb487568e8d128] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/config/systemInfo client=114.221.202.171, traceId=8c4eb1e8ed8d04dd, duration=4 ms] 2025-09-02 13:35:44.925 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,65846bcaccaf3020,23b93b81439e5b63] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/iam/actionGroups?null, client=114.221.202.171, traceId=65846bcaccaf3020, method=IterableResponse com.oceanbase.ocp.server.common.controller.IamController.listActionGroups(), args=] 2025-09-02 13:35:44.928 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,65846bcaccaf3020,23b93b81439e5b63] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/iam/actionGroups client=114.221.202.171, traceId=65846bcaccaf3020, duration=7 ms] 2025-09-02 13:35:45.101 INFO 13484 --- [prometheus-second-schedule-1,6f0968e4923afdde,696cd07de2e8f7a3] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=15, collectTaskCount=15 2025-09-02 13:35:45.608 INFO 13484 --- [ocp-schedules-13,e6188170052a415e,fdb33d1aa55fc9a5] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:35:45.623 WARN 13484 --- [scanner-12207784-1,e6188170052a415e,1f6f6f98403f45f2] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:35:46.323 INFO 13484 --- [prometheus-minute-schedule-1,c935cf3e6e4a5545,b97561049dd2cc15] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791360, interval=60 2025-09-02 13:35:46.327 INFO 13484 --- [prometheus-minute-schedule-1,c935cf3e6e4a5545,b97561049dd2cc15] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:35:46.420 INFO 13484 --- [ocp-async-3,cdb0847e2ae114c9,4a695e24ed406634] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:46.928 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:46.929 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:46.985 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:35:46.985128+08:00, lastProcessTime=2025-09-02T13:35:36.985970+08:00 2025-09-02 13:35:47.150 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:35:46.985128+08:00 2025-09-02 13:35:48.221 INFO 13484 --- [ocp-schedules-15,e7f2f1099f2d86d3,0cbd721bc8c7f7a3] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:35:48.421 INFO 13484 --- [ocp-async-6,f6cfc137b39de786,a620f69ccf30e7c8] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:48.850 INFO 13484 --- [ocp-schedules-20,20eb9922c78e2959,018f25532d5ccbb3] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster zone cgroup value. 2025-09-02 13:35:48.852 INFO 13484 --- [ocp-schedules-20,20eb9922c78e2959,018f25532d5ccbb3] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:35:48.879 INFO 13484 --- [Check cluster cgroup value.1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:48.879 INFO 13484 --- [Check cluster cgroup value.1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [enable_cgroup] 2025-09-02 13:35:48.891 INFO 13484 --- [ocp-schedules-20,20eb9922c78e2959,018f25532d5ccbb3] .o.o.s.t.b.c.AbstractShardingClusterTask : Check cluster cgroup value. executed completed, allCount=1, successCount=1 2025-09-02 13:35:49.092 INFO 13484 --- [ocp-async-5,e9c1992a93bca646,899e38183530c644] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:35:49.323 INFO 13484 --- [prometheus-minute-schedule-1,bb157898f8f3f422,0e01a3e6b3ea0226] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791360, interval=60 2025-09-02 13:35:49.328 INFO 13484 --- [prometheus-minute-schedule-1,bb157898f8f3f422,0e01a3e6b3ea0226] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:35:49.929 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:49.930 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:50.421 INFO 13484 --- [ocp-async-9,00c5b29cf04f29dc,014e4d40d569a0c9] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:50.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,46ffaf2476b59971,7676bf7edbd35bad] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=46ffaf2476b59971, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:35:50.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,9f10e7548a5feb78,ed20f64b0db18abf] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=9f10e7548a5feb78, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:35:50.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,52575e6ea14761cc,dfe7526ee21ad95b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=52575e6ea14761cc, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:35:50.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,3e7d2ace8ec98d35,3ab5670bedd57179] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=3e7d2ace8ec98d35, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:35:50.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,9f10e7548a5feb78,ed20f64b0db18abf] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=9f10e7548a5feb78, duration=4 ms] 2025-09-02 13:35:50.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,3e7d2ace8ec98d35,3ab5670bedd57179] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=3e7d2ace8ec98d35, duration=4 ms] 2025-09-02 13:35:50.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,46ffaf2476b59971,7676bf7edbd35bad] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=46ffaf2476b59971, duration=4 ms] 2025-09-02 13:35:50.926 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,52575e6ea14761cc,dfe7526ee21ad95b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=52575e6ea14761cc, duration=6 ms] 2025-09-02 13:35:52.323 INFO 13484 --- [prometheus-minute-schedule-1,381c8722050c0358,06fb0725a5ef21b0] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791360, interval=60 2025-09-02 13:35:52.328 INFO 13484 --- [prometheus-minute-schedule-1,381c8722050c0358,06fb0725a5ef21b0] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:35:52.421 INFO 13484 --- [ocp-async-3,44aa932e0a40c498,6d9c20eabad386a4] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:52.657 INFO 13484 --- [ocp-schedules-23,a6d6775c1da07eb4,3ae64780e346a089] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:35:52.659 INFO 13484 --- [ocp-schedules-23,a6d6775c1da07eb4,3ae64780e346a089] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:35:52.930 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:52.931 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:54.421 INFO 13484 --- [ocp-async-6,7cec3b662804dfbb,12fe06dc3137e7d8] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:55.323 INFO 13484 --- [prometheus-minute-schedule-1,2e75871725d04f0b,318c208581419abf] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791360, interval=60 2025-09-02 13:35:55.327 INFO 13484 --- [prometheus-minute-schedule-1,2e75871725d04f0b,318c208581419abf] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:35:55.704 INFO 13484 --- [ocp-schedules-1,2b30f5ea7e24fef7,43b55d582449fb41] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:35:55.721 WARN 13484 --- [scanner-12207922-1,2b30f5ea7e24fef7,90db2ba4a87eab8d] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:35:55.742 INFO 13484 --- [alarm-detect-5,2b30f5ea7e24fef7,8816a2f441800b75] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 3016171, data is empty. 2025-09-02 13:35:55.748 INFO 13484 --- [alarm-detect-7,2b30f5ea7e24fef7,42151eba0b1c7265] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 3016171, data is empty. 2025-09-02 13:35:55.750 INFO 13484 --- [alarm-detect-11,2b30f5ea7e24fef7,e85259f951ee6a83] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 3016171, data is empty. 2025-09-02 13:35:55.783 INFO 13484 --- [alarm-detect-7,2b30f5ea7e24fef7,62b6dcf588fe5702] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 3016171, data is empty. 2025-09-02 13:35:55.931 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:55.934 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:56.421 INFO 13484 --- [ocp-async-4,da4cdf48d960c75d,9b983f5afcbe8439] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:56.539 INFO 13484 --- [ocp-schedules-28,5940d5a543345808,4bb176ce2049ec4d] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-09-02 13:35:56.552 INFO 13484 --- [ocp-schedules-16,2fd7cfcb9fba951a,3afe838ed4db55df] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:35:56.552 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=172.19.62.154, originalStatus=ONLINE 2025-09-02 13:35:56.552 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=172.19.62.156, originalStatus=ONLINE 2025-09-02 13:35:56.553 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=172.17.172.84, originalStatus=ONLINE 2025-09-02 13:35:56.553 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=172.17.172.86, originalStatus=ONLINE 2025-09-02 13:35:56.553 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=172.17.172.85, originalStatus=ONLINE 2025-09-02 13:35:56.556 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:35:56.556 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:35:56.556 INFO 13484 --- [ocp-schedules-23,25874744adb5dbe6,84eb8e5e0d5516d2] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:35:56.557 INFO 13484 --- [host-check-agent-status3,25874744adb5dbe6,253c639ba33244b6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=172.17.172.84, originalStatus=ONLINE 2025-09-02 13:35:56.557 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:35:56.557 INFO 13484 --- [host-check-agent-status1,25874744adb5dbe6,c461647e8a39b78d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=172.19.62.154, originalStatus=ONLINE 2025-09-02 13:35:56.557 INFO 13484 --- [host-check-agent-status2,25874744adb5dbe6,3e47c659db1806c6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=172.19.62.156, originalStatus=ONLINE 2025-09-02 13:35:56.557 INFO 13484 --- [host-check-agent-status5,25874744adb5dbe6,4c43dfe42b32aa1c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=172.17.172.85, originalStatus=ONLINE 2025-09-02 13:35:56.558 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:35:56.558 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:35:56.558 INFO 13484 --- [host-check-agent-status4,25874744adb5dbe6,3943c1495400d1c5] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=172.17.172.86, originalStatus=ONLINE 2025-09-02 13:35:56.558 INFO 13484 --- [ocp-schedules-19,089f457e0c5df607,f41792e1684a2ad4] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-09-02 13:35:56.559 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:35:56.560 INFO 13484 --- [ocp-schedules-19,089f457e0c5df607,f41792e1684a2ad4] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:35:56.560 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:35:56.560 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:35:56.561 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:35:56.561 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:35:56.562 INFO 13484 --- [host-check-agent-status2,25874744adb5dbe6,3e47c659db1806c6] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:35:56.563 INFO 13484 --- [host-check-agent-status2,25874744adb5dbe6,3e47c659db1806c6] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:35:56.564 INFO 13484 --- [host-check-agent-status3,25874744adb5dbe6,253c639ba33244b6] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:35:56.565 INFO 13484 --- [host-check-agent-status1,25874744adb5dbe6,c461647e8a39b78d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:35:56.565 INFO 13484 --- [ocp-schedules-13,dc06c074c4d254ed,ad869076debf9c7d] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-09-02 13:35:56.565 INFO 13484 --- [ocp-schedules-10,36e1fda8f0e6b293,5bae4492b10fd933] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-09-02 13:35:56.566 INFO 13484 --- [host-check-agent-status3,25874744adb5dbe6,253c639ba33244b6] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:35:56.566 INFO 13484 --- [host-check-agent-status4,25874744adb5dbe6,3943c1495400d1c5] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:35:56.566 INFO 13484 --- [ocp-schedules-3,a501c99a73399621,e93f4a8585b29445] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:35:56.567 INFO 13484 --- [ocp-schedules-10,36e1fda8f0e6b293,5bae4492b10fd933] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:35:56.567 INFO 13484 --- [host-check-agent-status1,25874744adb5dbe6,c461647e8a39b78d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:35:56.567 INFO 13484 --- [ocp-schedules-13,dc06c074c4d254ed,ad869076debf9c7d] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:35:56.568 INFO 13484 --- [host-check-agent-status4,25874744adb5dbe6,3943c1495400d1c5] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:35:56.570 INFO 13484 --- [host-check-clockdiff1,a501c99a73399621,f55b69ead015846f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.19.62.154], timeoutMillis=20000 2025-09-02 13:35:56.571 INFO 13484 --- [host-check-clockdiff2,a501c99a73399621,a8ef0265cff30c54] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.19.62.156], timeoutMillis=20000 2025-09-02 13:35:56.571 INFO 13484 --- [host-check-clockdiff3,a501c99a73399621,9841021778c67f2a] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.84], timeoutMillis=20000 2025-09-02 13:35:56.572 INFO 13484 --- [host-check-clockdiff4,a501c99a73399621,3cdea983cee5c56a] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.86], timeoutMillis=20000 2025-09-02 13:35:56.573 INFO 13484 --- [ocp-schedules-13,dc06c074c4d254ed,ad869076debf9c7d] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=1, successCount=1 2025-09-02 13:35:56.579 INFO 13484 --- [ocp-schedules-21,aee7ac7ac13b8f2e,3a3b978576365fef] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-09-02 13:35:56.587 INFO 13484 --- [host-check-agent-status5,25874744adb5dbe6,4c43dfe42b32aa1c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:35:56.588 INFO 13484 --- [host-check-clockdiff5,a501c99a73399621,2477cc2ccc71504d] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.85], timeoutMillis=20000 2025-09-02 13:35:56.593 INFO 13484 --- [host-check-clockdiff4,a501c99a73399621,3cdea983cee5c56a] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791356 0 0 , error= 2025-09-02 13:35:56.600 INFO 13484 --- [host-check-agent-status5,25874744adb5dbe6,4c43dfe42b32aa1c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:35:56.597 INFO 13484 --- [host-check-clockdiff5,a501c99a73399621,2477cc2ccc71504d] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791356 -1 -1 , error= 2025-09-02 13:35:56.605 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62888/api/v1/time, params:null 2025-09-02 13:35:56.605 INFO 13484 --- [host-check-agent-status2,25874744adb5dbe6,3e47c659db1806c6] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62888/api/v1/agent/status, params:null 2025-09-02 13:35:56.605 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62888/api/v1/time, params:null 2025-09-02 13:35:56.607 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62888/api/v1/time, params:null 2025-09-02 13:35:56.607 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=172.17.172.86, timestamp=2025-09-02T13:35:56.606737038+08:00 2025-09-02 13:35:56.607 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=172.17.172.84, timestamp=2025-09-02T13:35:56.607089573+08:00 2025-09-02 13:35:56.608 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-09-02T13:35:56.607+08:00 2025-09-02 13:35:56.608 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-09-02T13:35:56.607+08:00 2025-09-02 13:35:56.610 INFO 13484 --- [host-check-agent-status2,25874744adb5dbe6,3e47c659db1806c6] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.19.62.156 2025-09-02 13:35:56.610 INFO 13484 --- [host-check-agent-status2,25874744adb5dbe6,3e47c659db1806c6] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.19.62.156 2025-09-02 13:35:56.610 INFO 13484 --- [host-check-agent-status2,25874744adb5dbe6,3e47c659db1806c6] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.19.62.156 2025-09-02 13:35:56.611 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62888/api/v1/time, params:null 2025-09-02 13:35:56.611 INFO 13484 --- [host-check-agent-status4,25874744adb5dbe6,3943c1495400d1c5] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62888/api/v1/agent/status, params:null 2025-09-02 13:35:56.611 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=172.19.62.156, timestamp=2025-09-02T13:35:56.607723546+08:00 2025-09-02 13:35:56.612 INFO 13484 --- [host-check-agent-status1,25874744adb5dbe6,c461647e8a39b78d] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62888/api/v1/agent/status, params:null 2025-09-02 13:35:56.612 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-09-02T13:35:56.611+08:00 2025-09-02 13:35:56.613 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=172.17.172.85, timestamp=2025-09-02T13:35:56.612176531+08:00 2025-09-02 13:35:56.613 INFO 13484 --- [host-check-agent-status4,25874744adb5dbe6,3943c1495400d1c5] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.86 2025-09-02 13:35:56.613 INFO 13484 --- [host-check-agent-status4,25874744adb5dbe6,3943c1495400d1c5] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.86 2025-09-02 13:35:56.613 INFO 13484 --- [host-check-agent-status4,25874744adb5dbe6,3943c1495400d1c5] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.86 2025-09-02 13:35:56.614 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62888/api/v1/time, params:null 2025-09-02 13:35:56.614 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-09-02T13:35:56.613+08:00 2025-09-02 13:35:56.616 INFO 13484 --- [host-check-agent-status3,25874744adb5dbe6,253c639ba33244b6] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62888/api/v1/agent/status, params:null 2025-09-02 13:35:56.616 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=172.19.62.154, timestamp=2025-09-02T13:35:56.615779603+08:00 2025-09-02 13:35:56.616 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-09-02T13:35:56.616+08:00 2025-09-02 13:35:56.617 INFO 13484 --- [host-check-agent-status1,25874744adb5dbe6,c461647e8a39b78d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.19.62.154 2025-09-02 13:35:56.617 INFO 13484 --- [host-check-agent-status1,25874744adb5dbe6,c461647e8a39b78d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.19.62.154 2025-09-02 13:35:56.617 INFO 13484 --- [host-check-agent-status1,25874744adb5dbe6,c461647e8a39b78d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.19.62.154 2025-09-02 13:35:56.618 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-09-02T13:35:56.607+08:00 and status AVAILABLE 2025-09-02 13:35:56.618 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-09-02T13:35:56.616+08:00 and status AVAILABLE 2025-09-02 13:35:56.618 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-09-02T13:35:56.607+08:00 and status AVAILABLE 2025-09-02 13:35:56.619 INFO 13484 --- [host-check-agent-status3,25874744adb5dbe6,253c639ba33244b6] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.84 2025-09-02 13:35:56.619 INFO 13484 --- [host-check-agent-status3,25874744adb5dbe6,253c639ba33244b6] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.84 2025-09-02 13:35:56.619 INFO 13484 --- [host-check-agent-status3,25874744adb5dbe6,253c639ba33244b6] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.84 2025-09-02 13:35:56.621 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-09-02T13:35:56.611+08:00 and status AVAILABLE 2025-09-02 13:35:56.622 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-09-02T13:35:56.613+08:00 and status AVAILABLE 2025-09-02 13:35:56.623 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-09-02 13:35:56.623 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-09-02 13:35:56.624 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-09-02 13:35:56.625 INFO 13484 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.626 INFO 13484 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-09-02 13:35:56.627 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-09-02 13:35:56.629 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-09-02 13:35:56.630 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:35:56.630 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:35:56.631 INFO 13484 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.3.5.3 2025-09-02 13:35:56.631 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:35:56.631 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:35:56.631 INFO 13484 --- [ocp-schedules-19,089f457e0c5df607,f41792e1684a2ad4] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:35:56.631 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:35:56.632 INFO 13484 --- [host-check-agent-status5,25874744adb5dbe6,4c43dfe42b32aa1c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62888/api/v1/agent/status, params:null 2025-09-02 13:35:56.633 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:35:56.634 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:35:56.636 INFO 13484 --- [host-check-agent-status5,25874744adb5dbe6,4c43dfe42b32aa1c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.85 2025-09-02 13:35:56.636 INFO 13484 --- [host-check-agent-status5,25874744adb5dbe6,4c43dfe42b32aa1c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.85 2025-09-02 13:35:56.636 INFO 13484 --- [host-check-agent-status5,25874744adb5dbe6,4c43dfe42b32aa1c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.85 2025-09-02 13:35:56.639 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:35:56.641 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:35:56.641 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:35:56.645 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.645 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-09-02 13:35:56.652 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:35:56.655 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:35:56.655 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:35:56.659 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:35:56.661 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:35:56.674 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-09-02 13:35:56.676 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-09-02 13:35:56.680 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:35:56.681 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.682 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-09-02 13:35:56.683 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-09-02 13:35:56.684 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-09-02 13:35:56.686 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-09-02 13:35:56.693 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-09-02 13:35:56.710 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:35:56.711 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:35:56.715 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:35:56.721 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:35:56.723 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.724 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-09-02 13:35:56.730 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-09-02 13:35:56.732 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.734 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-09-02 13:35:56.735 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-09-02 13:35:56.736 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-09-02 13:35:56.737 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-09-02 13:35:56.739 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:35:56.740 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-09-02 13:35:56.741 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-09-02 13:35:56.748 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-09-02 13:35:56.748 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-09-02 13:35:56.748 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-09-02 13:35:56.751 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-09-02 13:35:56.760 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:35:56.760 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.025724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:35:56.764 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=BackupAlarmCheckSchedule, method=run 2025-09-02 13:35:56.765 INFO 13484 --- [ocp-schedules-14,f60afd081a4b6644,4dc5e497ef8240f2] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2025-09-02 13:35:56.767 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:35:56.771 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:35:56.774 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:35:56.775 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.b.i.alarm.BackupAlarmService : Backup alarm check schedule. 2025-09-02 13:35:56.776 INFO 13484 --- [ocp-schedules-14,f60afd081a4b6644,4dc5e497ef8240f2] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2025-09-02 13:35:56.779 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:35:56.782 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:35:56.785 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.025724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:35:56.785 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:35:56.795 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-09-02 13:35:56.800 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-09-02 13:35:56.800 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:35:56.800 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27.368133+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:35:56.804 INFO 13484 --- [host-check-host-status3,2fd7cfcb9fba951a,4c187e7097a8c5f0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-09-02 13:35:56.806 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-09-02 13:35:56.810 INFO 13484 --- [host-check-host-status4,2fd7cfcb9fba951a,1c67f0f5b7610fd7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-09-02 13:35:56.815 INFO 13484 --- [host-check-host-status5,2fd7cfcb9fba951a,c20c8f4eb8cf65d5] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-09-02 13:35:56.825 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.826 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:35:56.827 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.828 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:35:56.834 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27.368133+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:35:56.834 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:35:56.840 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.847 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-09-02 13:35:56.851 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:35:56.851 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.373150+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:35:56.854 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-09-02 13:35:56.857 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2025-09-02 13:35:56.860 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-09-02 13:35:56.866 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-09-02 13:35:56.875 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.373150+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:35:56.881 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:35:56.885 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:35:56.890 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:35:56.891 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.892 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-09-02 13:35:56.896 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster myoceanbase 2025-09-02 13:35:56.900 INFO 13484 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 3, sqlPort: 2881 2025-09-02 13:35:56.905 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:35:56.905 INFO 13484 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:35:56.907 INFO 13484 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:35:56.907 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.908 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:35:56.909 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.910 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:35:56.921 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.938 INFO 13484 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-09-02 13:35:56.946 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2025-09-02 13:35:56.955 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-09-02 13:35:56.956 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-09-02 13:35:56.965 INFO 13484 --- [host-check-host-status2,2fd7cfcb9fba951a,08047469194b6452] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-09-02 13:35:56.965 INFO 13484 --- [host-check-host-status1,2fd7cfcb9fba951a,3832eb0e98505bc8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-09-02 13:35:56.966 INFO 13484 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 3, sqlPort: 2881 got ObProcessInfoResult(pid=4794, runPath=/data1/myoceanbase/oceanbase, runUser=root, execute=/data1/myoceanbase/oceanbase/bin/observer, installPath=/data1/myoceanbase/oceanbase) 2025-09-02 13:35:56.980 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2025-09-02 13:35:56.980 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:35:56.986 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:35:56.985049+08:00, lastProcessTime=2025-09-02T13:35:46.985128+08:00 2025-09-02 13:35:56.987 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:56.988 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-09-02 13:35:57.004 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=myoceanbase, obClusterId=1755130302 2025-09-02 13:35:57.004 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=myoceanbase, obClusterId=1755130302, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"myoceanbase","ObClusterId":1755130302,"ObRegion":"myoceanbase","ObRegionId":1755130302,"ReadonlyRsList":[],"RsList":[{"address":"172.17.172.85:2882","role":"LEADER","sql_port":2881},{"address":"172.17.172.86:2882","role":"FOLLOWER","sql_port":2881},{"address":"172.17.172.84:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1756791174423000}, createTime=2025-08-29T16:02:30+08:00, creator=admin, obVersion=4.3.5.3, packageOperatingSystem=null, loadType=null, attributesJson={"DataDiskPath":"/data1/myoceanbase/oceanbase/store","DiskPathStyle":"OBD","InstallPath":"/data1/myoceanbase/oceanbase","LogDiskPath":"/data1/myoceanbase/oceanbase/store","OperatingSystemUser":"root","RunPath":"/data1/myoceanbase/oceanbase","SqlPort":2881,"SvrPort":2882}, tenantCount=2) 2025-09-02 13:35:57.004 INFO 13484 --- [ocp-schedules-10,36e1fda8f0e6b293,5bae4492b10fd933] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:35:57.011 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:35:57.018 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:57.018 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:35:57.020 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:57.020 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:35:57.032 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:57.038 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [600] 2025-09-02 13:35:57.063 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:35:57.065 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:57.065 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2025-09-02 13:35:57.066 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2025-09-02 13:35:57.082 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.b.i.alarm.BackupAlarmService : clusterId=1, minDelaySeconds=600, delayTasks=[] 2025-09-02 13:35:57.106 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:35:57.108 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:57.108 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:35:57.110 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:57.110 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:35:57.122 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:57.138 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [60] 2025-09-02 13:35:57.143 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:35:56.985049+08:00 2025-09-02 13:35:57.175 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=60, timeoutTasks=[] 2025-09-02 13:35:57.201 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:35:57.203 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:57.203 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:35:57.205 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:57.205 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:35:57.216 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:35:57.264 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2025-09-02 13:35:57.296 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2025-09-02 13:35:57.309 INFO 13484 --- [ocp-schedules-12,823bd6655d02351b,f1e843573e999f55] c.o.o.b.i.alarm.BackupAlarmService : cluster=myoceanbase does not enable secondary backup 2025-09-02 13:35:57.488 INFO 13484 --- [ocp-schedules-5,6fe7a58c4f14b1da,3fbcdf5f02a4f336] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:35:57.488 INFO 13484 --- [ocp-schedules-5,6fe7a58c4f14b1da,3fbcdf5f02a4f336] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2025-09-02 13:35:57.488 INFO 13484 --- [ForkJoinPool.commonPool-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2025-09-02 13:35:57.488 INFO 13484 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2025-09-02 13:35:57.488 INFO 13484 --- [ForkJoinPool.commonPool-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2025-09-02 13:35:57.488 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2025-09-02 13:35:57.493 INFO 13484 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:35:57.493 INFO 13484 --- [ocp-schedules-5,6fe7a58c4f14b1da,3fbcdf5f02a4f336] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:35:57.493 INFO 13484 --- [ForkJoinPool.commonPool-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:35:57.494 INFO 13484 --- [ForkJoinPool.commonPool-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:35:57.494 INFO 13484 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:35:57.494 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:35:57.495 INFO 13484 --- [ocp-schedules-5,6fe7a58c4f14b1da,3fbcdf5f02a4f336] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:35:57.495 INFO 13484 --- [ForkJoinPool.commonPool-worker-6,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:35:57.496 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:35:57.496 INFO 13484 --- [ForkJoinPool.commonPool-worker-0,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:35:57.513 INFO 13484 --- [ForkJoinPool.commonPool-worker-0,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62889/api/v1/log/alarms, params:null 2025-09-02 13:35:57.528 INFO 13484 --- [host-check-clockdiff1,a501c99a73399621,f55b69ead015846f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791357 0 0 , error= 2025-09-02 13:35:57.526 INFO 13484 --- [host-check-clockdiff3,a501c99a73399621,9841021778c67f2a] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791357 -1 -1 , error= 2025-09-02 13:35:57.529 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62889/api/v1/log/alarms, params:null 2025-09-02 13:35:57.530 INFO 13484 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62889/api/v1/log/alarms, params:null 2025-09-02 13:35:57.533 INFO 13484 --- [ForkJoinPool.commonPool-worker-6,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62889/api/v1/log/alarms, params:null 2025-09-02 13:35:57.535 INFO 13484 --- [ocp-schedules-5,6fe7a58c4f14b1da,3fbcdf5f02a4f336] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62889/api/v1/log/alarms, params:null 2025-09-02 13:35:57.537 INFO 13484 --- [ocp-schedules-5,6fe7a58c4f14b1da,3fbcdf5f02a4f336] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms finished. 2025-09-02 13:35:57.538 INFO 13484 --- [print-time-series-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Async compute queue max size = 0, current size = 0. 2025-09-02 13:35:57.543 INFO 13484 --- [ocp-schedules-9,a5adc86c3168970c,bdd2cc8a26108149] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":10713,"minuteWriteQueue":0,"minuteReadCacheSize":4820,"secondWriteCacheSize":2678,"seriesIdKeySize":5749,"secondWriteQueue":0,"minuteWriteCacheSize":4806} 2025-09-02 13:35:57.544 INFO 13484 --- [ocp-schedules-9,a5adc86c3168970c,bdd2cc8a26108149] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{ob_sysstat_40012=15, ob_server_resource_memory_assigned_bytes=10, ob_sysstat_40011=15, ob_sysstat_40010=15, ob_unit_num=10, ob_database_disk_required_size=29, go_memstats_mspan_sys_bytes=16, ob_sysstat_30213=15, go_memstats_stack_sys_bytes=16, ob_sysstat_30212=15, ob_sysstat_30217=15, ob_sysstat_30216=15, ob_tenant_request_queue_total_size=16, jvm_gc_memory_allocated_bytes_total=2, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=10, ob_sysstat_40019=15, ob_sysstat_40018=15, processor_process_metrics_total=20, jvm_gc_max_data_size_bytes=2, ob_sysstat_40027=15, ob_sysstat_40026=15, ob_sysstat_40025=15, go_memstats_frees_total=16, host_available=15, ob_sysstat_30220=15, ob_sysstat_30222=15, ob_sysstat_30221=15, mysql_output_write_sql_failed_count=19, ob_database_disk_data_size=28, ob_tenant_memstore_freeze_cnt=12, ob_tenant_unit_min_memory=21, ob_sysstat_40028=15, ob_sysstat_60087=15, ob_memstore_total_bytes=16, jvm_threads_states_threads=8, process_resident_memory_bytes=16, ob_cache_size_bytes=33, ob_tenant_unit_max_memory=21, input_collect_metrics_total=24, ob_sysstat_50039=15, ob_sysstat_60083=15, ob_sysstat_50038=15, node_disk_io_time_weighted_seconds_total=27, process_open_fds=16, ob_sysstat_50037=15, process_memory_rss_bytes=29, ob_sysstat_50030=15, ob_server_resource_cpu_assigned_percent=10, sql_audit_input_collect_lost_total=12, go_memstats_heap_inuse_bytes=16, ob_sysstat_40118=15, ob_sysstat_40117=15, node_memory_memfree_bytes=20, ob_sysstat_40116=15, ob_sysstat_40005=15, process_exists=39, ob_sysstat_40004=15, ob_sysstat_40003=15, ob_sysstat_40002=15, go_gc_duration_seconds=21, ob_server_resource_disk_bytes=10, ob_sysstat_40001=15, ob_sysstat_40000=15, mysql_output_write_sql_time=19, ob_sysstat_30200=15, odp_current_session=17, ob_sysstat_30201=15, ob_sysstat_30204=15, ob_sysstat_30205=15, ob_sysstat_30208=15, process_thread_count=29, ob_sysstat_30209=15, ob_sysstat_50040=15, ob_sysstat_40009=15, ob_disk_free_bytes=10, ob_sysstat_40008=15, node_filesystem_avail_bytes=49, 2025-09-02 13:35:58.226 INFO 13484 --- [ocp-schedules-6,29256655a10bad93,186f69a2fcfe05be] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:35:58.323 INFO 13484 --- [prometheus-minute-schedule-1,a5dd1d371f489508,d6764932999db424] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791360, interval=60 2025-09-02 13:35:58.327 INFO 13484 --- [prometheus-minute-schedule-1,a5dd1d371f489508,d6764932999db424] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:35:58.360 INFO 13484 --- [host-check-clockdiff2,a501c99a73399621,a8ef0265cff30c54] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791358 -1 -1 , error= 2025-09-02 13:35:58.421 INFO 13484 --- [ocp-async-5,a89f5a8ceec1a708,d9e8aa3e98a05dfe] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:35:58.939 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:35:58.940 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:35:59.092 INFO 13484 --- [ocp-async-7,e18839604c9835e4,2287539e24309572] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:36:00.422 INFO 13484 --- [ocp-async-2,1b490989e1a8bf1b,1f29c9b0428b0631] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:00.734 INFO 13484 --- [ocp-schedules-19,2f9d637d9d93f0c0,ea5ae15c72809edf] c.o.o.s.c.scheduled.TenantSchedules : Collect all tenant major compaction 2025-09-02 13:36:00.734 INFO 13484 --- [ocp-schedules-19,2f9d637d9d93f0c0,ea5ae15c72809edf] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2025-09-02 13:36:00.749 INFO 13484 --- [Collect all tenants major compactions1,2f9d637d9d93f0c0,1aef92ec80e92196] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:myoceanbase, tenant:woqu 2025-09-02 13:36:00.749 INFO 13484 --- [Collect all tenants major compactions2,2f9d637d9d93f0c0,0f85dbfec37cd0d2] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:myoceanbase, tenant:sys 2025-09-02 13:36:00.776 INFO 13484 --- [Collect all tenants major compactions2,2f9d637d9d93f0c0,0f85dbfec37cd0d2] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:00.776 INFO 13484 --- [Collect all tenants major compactions1,2f9d637d9d93f0c0,1aef92ec80e92196] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:00.777 INFO 13484 --- [Collect all tenants major compactions2,2f9d637d9d93f0c0,0f85dbfec37cd0d2] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-09-02 13:36:00.778 INFO 13484 --- [Collect all tenants major compactions1,2f9d637d9d93f0c0,1aef92ec80e92196] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2025-09-02 13:36:00.787 INFO 13484 --- [Collect all tenants major compactions1,2f9d637d9d93f0c0,1aef92ec80e92196] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:myoceanbase, tenant:woqu 2025-09-02 13:36:00.789 INFO 13484 --- [Collect all tenants major compactions2,2f9d637d9d93f0c0,0f85dbfec37cd0d2] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:myoceanbase, tenant:sys 2025-09-02 13:36:00.789 INFO 13484 --- [ocp-schedules-19,2f9d637d9d93f0c0,ea5ae15c72809edf] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2025-09-02 13:36:00.883 INFO 13484 --- [ocp-schedules-23,b13f96a58c2ff490,1371960509321068] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog instances, size=0 2025-09-02 13:36:00.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,2a2acbc09051c93d,be4a7fe5ab221c4b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=2a2acbc09051c93d, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:36:00.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,d990a406608c0f3a,04f425f0742d6804] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=d990a406608c0f3a, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:36:00.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,6af721c5d13a1f84,ec5568ec663d9016] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=6af721c5d13a1f84, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:36:00.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,268209c964eb94e2,ea13581d6f77c803] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=268209c964eb94e2, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:36:00.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,2a2acbc09051c93d,be4a7fe5ab221c4b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=2a2acbc09051c93d, duration=4 ms] 2025-09-02 13:36:00.936 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,d990a406608c0f3a,04f425f0742d6804] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=d990a406608c0f3a, duration=5 ms] 2025-09-02 13:36:00.936 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,6af721c5d13a1f84,ec5568ec663d9016] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=6af721c5d13a1f84, duration=5 ms] 2025-09-02 13:36:00.940 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,268209c964eb94e2,ea13581d6f77c803] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=268209c964eb94e2, duration=9 ms] 2025-09-02 13:36:01.020 INFO 13484 --- [ocp-schedules-8,ef4bd1f4abecf9ac,f6d3c4940587b256] c.o.o.s.c.scheduled.ClusterSchedules : Collect all cluster compaction info. 2025-09-02 13:36:01.022 INFO 13484 --- [ocp-schedules-8,ef4bd1f4abecf9ac,f6d3c4940587b256] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:01.022 INFO 13484 --- [Collect compaction for all clusters1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-09-02 13:36:01.022 INFO 13484 --- [ocp-schedules-8,ef4bd1f4abecf9ac,f6d3c4940587b256] .o.o.s.t.b.c.AbstractShardingClusterTask : Collect compaction for all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:36:01.323 INFO 13484 --- [prometheus-minute-schedule-1,fa983f9b86f7cb17,110600f858b7f9f3] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:01.327 INFO 13484 --- [prometheus-minute-schedule-1,fa983f9b86f7cb17,110600f858b7f9f3] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:01.940 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:01.941 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:02.060 INFO 13484 --- [ocp-schedules-15,36a94932655dcbbb,e886b24d677df0e9] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info 2025-09-02 13:36:02.064 INFO 13484 --- [ocp-schedules-15,36a94932655dcbbb,e886b24d677df0e9] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Invalidate clusters [] cache info success. 2025-09-02 13:36:02.064 INFO 13484 --- [ocp-schedules-15,36a94932655dcbbb,e886b24d677df0e9] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info success, cached size: 1 2025-09-02 13:36:02.422 INFO 13484 --- [ocp-async-10,a6eb00e2d0916747,9ebde0a43252d029] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:02.668 INFO 13484 --- [ocp-schedules-8,3ad400e1ba93fb0e,e0d5ea329dfb85b6] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:36:02.671 INFO 13484 --- [ocp-schedules-8,3ad400e1ba93fb0e,e0d5ea329dfb85b6] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:36:03.713 INFO 13484 --- [ocp-schedules-17,b47104087535c4a9,bd8ba4070851a2da] c.o.o.s.c.scheduled.TenantSchedules : Check all tenant service name conflict 2025-09-02 13:36:03.713 INFO 13484 --- [ocp-schedules-17,b47104087535c4a9,bd8ba4070851a2da] .t.CheckAllTenantServiceNameConflictTask : Check all tenant service name conflict. 2025-09-02 13:36:03.730 INFO 13484 --- [ocp-schedules-22,75a82e6cdbba3886,1a884ca7b3fc5dc7] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-09-02 13:36:03.737 INFO 13484 --- [ocp-schedules-22,75a82e6cdbba3886,1a884ca7b3fc5dc7] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:36:03.737 INFO 13484 --- [ocp-schedules-22,75a82e6cdbba3886,1a884ca7b3fc5dc7] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:36:03.944 INFO 13484 --- [ocp-schedules-21,059016e94a0ef3a7,28b8494b791e7212] c.o.o.s.c.scheduled.TenantSchedules : Check suspend tenant compaction status 2025-09-02 13:36:03.944 INFO 13484 --- [ocp-schedules-21,059016e94a0ef3a7,28b8494b791e7212] s.t.b.t.CheckSuspendTenantCompactionTask : Check tenant compaction status. 2025-09-02 13:36:03.987 INFO 13484 --- [ocp-schedules-21,059016e94a0ef3a7,28b8494b791e7212] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:03.988 INFO 13484 --- [ocp-schedules-21,059016e94a0ef3a7,28b8494b791e7212] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-09-02 13:36:04.323 INFO 13484 --- [prometheus-minute-schedule-1,2f2e69f3a3b87143,171ddca44c80144d] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:04.327 INFO 13484 --- [prometheus-minute-schedule-1,2f2e69f3a3b87143,171ddca44c80144d] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:04.422 INFO 13484 --- [ocp-async-5,4ef9957d8984891b,65fa0c9e3760d4f3] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:04.917 INFO 13484 --- [ocp-schedules-27,5fcafab55e6db3c5,2744c6b211f5d77b] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-09-02 13:36:04.941 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:04.942 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:05.507 INFO 13484 --- [ocp-schedules-9,5a64a8f6a86775fe,960c1174ce6d4ef4] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2025-09-02 13:36:05.507 INFO 13484 --- [ocp-schedules-9,5a64a8f6a86775fe,960c1174ce6d4ef4] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=819357824821161984 2025-09-02 13:36:05.512 INFO 13484 --- [ocp-schedules-9,5a64a8f6a86775fe,960c1174ce6d4ef4] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2025-09-02 13:36:05.512 INFO 13484 --- [ocp-schedules-9,5a64a8f6a86775fe,960c1174ce6d4ef4] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=818961159966617600 2025-09-02 13:36:05.516 INFO 13484 --- [ocp-schedules-9,5a64a8f6a86775fe,960c1174ce6d4ef4] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2025-09-02 13:36:05.516 INFO 13484 --- [ocp-schedules-9,5a64a8f6a86775fe,960c1174ce6d4ef4] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=818784736907816960 2025-09-02 13:36:05.804 INFO 13484 --- [ocp-schedules-20,3761bf86bf8cb823,be765c8c23a4232f] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:36:05.819 WARN 13484 --- [scanner-12208088-1,3761bf86bf8cb823,4f3fae670e5bd603] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:36:06.422 INFO 13484 --- [ocp-async-8,137fd9697f833a1a,3236a437887ff315] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:06.987 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:36:06.986163+08:00, lastProcessTime=2025-09-02T13:35:56.985049+08:00 2025-09-02 13:36:07.137 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:36:06.986163+08:00 2025-09-02 13:36:07.323 INFO 13484 --- [prometheus-minute-schedule-1,3fc6e580d33a690c,e2d0b4e2f168634a] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:07.328 INFO 13484 --- [prometheus-minute-schedule-1,3fc6e580d33a690c,e2d0b4e2f168634a] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:07.942 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:07.943 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:08.230 INFO 13484 --- [ocp-schedules-19,0d3bc07f77518eea,e64cee48b67c8335] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:36:08.423 INFO 13484 --- [ocp-async-10,9622efcb17790bb5,52962fcaac04670a] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:08.622 INFO 13484 --- [ocp-schedules-29,52ffc4f32e182449,01be1c34519be459] c.o.o.s.c.scheduled.ClusterSchedules : Sync root server list. 2025-09-02 13:36:08.624 INFO 13484 --- [ocp-schedules-29,52ffc4f32e182449,01be1c34519be459] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:08.637 INFO 13484 --- [ocp-schedules-29,52ffc4f32e182449,01be1c34519be459] .o.o.s.t.b.c.AbstractShardingClusterTask : Refresh root service list executed completed, allCount=1, successCount=1 2025-09-02 13:36:09.093 INFO 13484 --- [ocp-async-4,0ece3b800936f439,09471304746d6b74] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:36:09.288 INFO 13484 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:09.288 INFO 13484 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-09-02 13:36:09.290 INFO 13484 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:09.290 INFO 13484 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-09-02 13:36:09.292 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule start 2025-09-02 13:36:09.304 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING} 2025-09-02 13:36:09.317 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (myoceanbase:1755130302-1) 2025-09-02 13:36:09.317 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (myoceanbase:1755130302-2) 2025-09-02 13:36:09.317 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (myoceanbase:1755130302-3) 2025-09-02 13:36:09.323 INFO 13484 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:09.323 INFO 13484 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:09.323 INFO 13484 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:09.323 INFO 13484 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-09-02 13:36:09.324 INFO 13484 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-09-02 13:36:09.324 INFO 13484 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2025-09-02 13:36:09.325 INFO 13484 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:09.325 INFO 13484 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:09.325 INFO 13484 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ now() 2025-09-02 13:36:09.325 INFO 13484 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ now() 2025-09-02 13:36:09.325 INFO 13484 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:09.326 INFO 13484 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ now() 2025-09-02 13:36:09.373 INFO 13484 --- [sql-rollup-worker-0,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=myoceanbase:1755130302, server=3, level=0->1, c=0) - source=2025-09-02T13:33:30, start=2025-09-02T13:31:00, end=2025-09-02T13:33:30, updates=308, granule=120000ms, delay=150000ms 2025-09-02 13:36:09.383 INFO 13484 --- [sql-rollup-worker-0,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=3, level=0->1, c=1) - not enough cached interval (source=2025-09-02T13:33:30, target=2025-09-02T13:33:30, granule=120000ms, delay=0ms) 2025-09-02 13:36:09.414 INFO 13484 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=myoceanbase:1755130302, server=2, level=0->1, c=0) - source=2025-09-02T13:33:30, start=2025-09-02T13:31:00, end=2025-09-02T13:33:30, updates=632, granule=120000ms, delay=150000ms 2025-09-02 13:36:09.420 INFO 13484 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=myoceanbase:1755130302, server=1, level=0->1, c=0) - source=2025-09-02T13:33:30, start=2025-09-02T13:31:00, end=2025-09-02T13:33:30, updates=983, granule=120000ms, delay=150000ms 2025-09-02 13:36:09.427 INFO 13484 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=2, level=0->1, c=1) - not enough cached interval (source=2025-09-02T13:33:30, target=2025-09-02T13:33:30, granule=120000ms, delay=0ms) 2025-09-02 13:36:09.435 INFO 13484 --- [sql-rollup-worker-0,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=myoceanbase:1755130302, server=3, level=1->2, c=0) - source=2025-09-02T13:33:30, start=2025-09-02T13:18:00, end=2025-09-02T13:28:00, updates=387, granule=600000ms, delay=930000ms 2025-09-02 13:36:09.438 INFO 13484 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=1, level=0->1, c=1) - not enough cached interval (source=2025-09-02T13:33:30, target=2025-09-02T13:33:30, granule=120000ms, delay=0ms) 2025-09-02 13:36:09.438 INFO 13484 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=1, level=1->2, c=0) - not enough cached interval (source=2025-09-02T13:33:30, target=2025-09-02T13:27:00, granule=600000ms, delay=390000ms) 2025-09-02 13:36:09.440 INFO 13484 --- [sql-rollup-worker-0,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=3, level=1->2, c=1) - not enough cached interval (source=2025-09-02T13:33:30, target=2025-09-02T13:28:00, granule=600000ms, delay=330000ms) 2025-09-02 13:36:09.513 INFO 13484 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=myoceanbase:1755130302, server=2, level=1->2, c=0) - source=2025-09-02T13:33:30, start=2025-09-02T13:17:30, end=2025-09-02T13:27:30, updates=610, granule=600000ms, delay=960000ms 2025-09-02 13:36:09.518 INFO 13484 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=2, level=1->2, c=1) - not enough cached interval (source=2025-09-02T13:33:30, target=2025-09-02T13:27:30, granule=600000ms, delay=360000ms) 2025-09-02 13:36:09.519 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule end, elapsed=227ms, period=120000ms 2025-09-02 13:36:10.004 INFO 13484 --- [ocp-schedules-29,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=2 2025-09-02 13:36:10.004 INFO 13484 --- [ocp-schedules-6,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=2 2025-09-02 13:36:10.021 INFO 13484 --- [Check obproxy1,b95bb674578172d3,50c102c85da5af13] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:36:10.021 INFO 13484 --- [Check obproxy2,a9af1b6a3de2e642,57260947bc33b09f] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:36:10.021 INFO 13484 --- [Check obproxy1,b95bb674578172d3,50c102c85da5af13] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:36:10.021 INFO 13484 --- [Check obproxy2,a9af1b6a3de2e642,57260947bc33b09f] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:36:10.024 INFO 13484 --- [Collect obproxy2,ce790e403f427d14,5c3a6792013d961c] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:36:10.024 INFO 13484 --- [Collect obproxy2,ce790e403f427d14,5c3a6792013d961c] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:36:10.024 INFO 13484 --- [Collect obproxy1,15baea4fd0c909a3,507f8f96cce6518a] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:36:10.024 INFO 13484 --- [Collect obproxy1,15baea4fd0c909a3,507f8f96cce6518a] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:36:10.045 INFO 13484 --- [Check obproxy2,a9af1b6a3de2e642,57260947bc33b09f] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:36:10.047 INFO 13484 --- [Check obproxy1,b95bb674578172d3,50c102c85da5af13] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:36:10.049 INFO 13484 --- [Check obproxy2,a9af1b6a3de2e642,57260947bc33b09f] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-09-02 13:36:10.049 INFO 13484 --- [Check obproxy2,a9af1b6a3de2e642,57260947bc33b09f] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 5, need to sync to meta: false 2025-09-02 13:36:10.079 INFO 13484 --- [Collect obproxy2,ce790e403f427d14,5c3a6792013d961c] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:36:10.079 INFO 13484 --- [Check obproxy1,b95bb674578172d3,50c102c85da5af13] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-09-02 13:36:10.079 INFO 13484 --- [Check obproxy1,b95bb674578172d3,50c102c85da5af13] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 6, need to sync to meta: false 2025-09-02 13:36:10.083 INFO 13484 --- [Collect obproxy2,ce790e403f427d14,5c3a6792013d961c] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:36:10.083 INFO 13484 --- [Collect obproxy1,15baea4fd0c909a3,507f8f96cce6518a] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:36:10.085 INFO 13484 --- [Collect obproxy2,ce790e403f427d14,5c3a6792013d961c] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:36:10.087 INFO 13484 --- [Collect obproxy1,15baea4fd0c909a3,507f8f96cce6518a] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:36:10.090 INFO 13484 --- [Collect obproxy1,15baea4fd0c909a3,507f8f96cce6518a] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:36:10.323 INFO 13484 --- [prometheus-minute-schedule-1,d2e82c99e8af9cea,74851291670438bc] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:10.328 INFO 13484 --- [prometheus-minute-schedule-1,d2e82c99e8af9cea,74851291670438bc] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:10.423 INFO 13484 --- [ocp-async-1,1430a91f1e4a6beb,e72f50a956790eea] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:10.554 INFO 13484 --- [async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 11, ruleSize = 11. 2025-09-02 13:36:10.764 INFO 13484 --- [metric-archive-async-7,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1756791300, nodeCount=214, affectRows=214 2025-09-02 13:36:10.764 INFO 13484 --- [metric-archive-async-5,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1756791300, nodeCount=169, affectRows=169 2025-09-02 13:36:10.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,920db65c7ba88ae0,394efe4dd9ddf23f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=920db65c7ba88ae0, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:36:10.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,2a6fffc5c8da38ec,ea326920fdc63cde] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=2a6fffc5c8da38ec, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:36:10.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,13427a31e704359d,15b739401355ef98] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=13427a31e704359d, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:36:10.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,920db65c7ba88ae0,394efe4dd9ddf23f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=920db65c7ba88ae0, duration=4 ms] 2025-09-02 13:36:10.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,6242db9ef2db99ca,d94da411c1417a4f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=6242db9ef2db99ca, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:36:10.925 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,6242db9ef2db99ca,d94da411c1417a4f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=6242db9ef2db99ca, duration=6 ms] 2025-09-02 13:36:10.925 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,13427a31e704359d,15b739401355ef98] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=13427a31e704359d, duration=6 ms] 2025-09-02 13:36:10.926 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,2a6fffc5c8da38ec,ea326920fdc63cde] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=2a6fffc5c8da38ec, duration=6 ms] 2025-09-02 13:36:10.943 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:10.944 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:12.423 INFO 13484 --- [ocp-async-8,36e14294ac49687c,d680f747f13f2758] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:12.678 INFO 13484 --- [ocp-schedules-17,4d88d0718f7c5429,569146c2330599fb] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:36:12.680 INFO 13484 --- [ocp-schedules-17,4d88d0718f7c5429,569146c2330599fb] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:36:13.323 INFO 13484 --- [prometheus-minute-schedule-1,39a1ab6d6be4f9c1,bc5fa0c03c46bd09] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:13.330 INFO 13484 --- [prometheus-minute-schedule-1,39a1ab6d6be4f9c1,bc5fa0c03c46bd09] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:13.501 INFO 13484 --- [metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1756791300, nodeCount=1000, affectRows=1000 2025-09-02 13:36:13.503 INFO 13484 --- [metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1756791300, nodeCount=1000, affectRows=1000 2025-09-02 13:36:13.504 INFO 13484 --- [metric-archive-second-8,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1756791300, nodeCount=678, affectRows=678 2025-09-02 13:36:13.506 INFO 13484 --- [metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1756791300, nodeCount=997, validCount=994 2025-09-02 13:36:13.508 INFO 13484 --- [metric-archive-second-8,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1756791300, nodeCount=676, validCount=674 2025-09-02 13:36:13.509 INFO 13484 --- [metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1756791300, nodeCount=997, validCount=996 2025-09-02 13:36:13.662 INFO 13484 --- [ocp-schedules-21,1509f452cfe6d85a,d3bbbc148c843e26] c.o.o.alarm.service.OcpAlarmServiceImpl : fix alarm target, alarms size 8 2025-09-02 13:36:13.664 INFO 13484 --- [ocp-schedules-9,d1ed2ab88fd5cbc3,9d4c7126c8bb9519] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration auto replace status 2025-09-02 13:36:13.667 INFO 13484 --- [ocp-schedules-14,305f91e3f14f2d8e,a50216f0f3161421] c.o.o.s.c.scheduled.ClusterSchedules : Check certificate expired. 2025-09-02 13:36:13.944 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:13.946 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:14.423 INFO 13484 --- [ocp-async-9,c2374d1ec57e58ae,f8b3cf4fc1c4b400] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:15.885 INFO 13484 --- [ocp-schedules-2,2d71efb3a1c1ace7,267660bc6cc2685f] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:36:15.934 WARN 13484 --- [scanner-12208888-1,2d71efb3a1c1ace7,e2804da79509a98c] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:36:15.942 INFO 13484 --- [alarm-detect-4,2d71efb3a1c1ace7,711c342265b06ace] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 3016171, data is empty. 2025-09-02 13:36:15.954 INFO 13484 --- [alarm-detect-10,2d71efb3a1c1ace7,cc35552f4f76caa3] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 3016171, data is empty. 2025-09-02 13:36:15.956 INFO 13484 --- [alarm-detect-7,2d71efb3a1c1ace7,dbbbed26da59e996] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 3016171, data is empty. 2025-09-02 13:36:16.018 INFO 13484 --- [alarm-detect-17,2d71efb3a1c1ace7,99f555e6656bbae6] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 3016171, data is empty. 2025-09-02 13:36:16.323 INFO 13484 --- [prometheus-minute-schedule-1,70d9858efaeec7ed,16318f839ff0934d] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:16.327 INFO 13484 --- [prometheus-minute-schedule-1,70d9858efaeec7ed,16318f839ff0934d] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:16.423 INFO 13484 --- [ocp-async-7,bd3324980914123f,91e827bcb43fdabe] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:16.946 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:16.947 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:16.985 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:36:16.985166+08:00, lastProcessTime=2025-09-02T13:36:06.986163+08:00 2025-09-02 13:36:17.149 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:36:16.985166+08:00 2025-09-02 13:36:18.090 INFO 13484 --- [ocp-schedules-22,af8b98d34bf94245,0a44866ac1eb0119] c.o.o.s.common.scheduled.PerfSchedules : Oas sql curing, clusters=[] 2025-09-02 13:36:18.235 INFO 13484 --- [ocp-schedules-17,13fd17a7a4a92574,f122bfaf537e856f] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:36:18.424 INFO 13484 --- [ocp-async-6,909a55df4b6fb6da,6ef6cbf3a3889941] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:19.093 INFO 13484 --- [ocp-async-1,1b0b1ad3524c8c73,3c7cc217b360fab3] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:36:19.323 INFO 13484 --- [prometheus-minute-schedule-1,6aaa7b9047349cf3,8887ece9b928f9c5] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:19.328 INFO 13484 --- [prometheus-minute-schedule-1,6aaa7b9047349cf3,8887ece9b928f9c5] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:19.947 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:19.948 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:20.424 INFO 13484 --- [ocp-async-10,73cc6acf762885de,e0b5ea86a5bce87c] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:20.669 INFO 13484 --- [ocp-schedules-16,35ff49847ea61cad,ae5c9e0e411d2eeb] c.o.o.s.common.scheduled.PerfSchedules : Collect trans stat for alarm, time=2025-09-02T13:36:20.669+08:00, clusters=[1] 2025-09-02 13:36:20.672 INFO 13484 --- [ocp-schedules-16,35ff49847ea61cad,ae5c9e0e411d2eeb] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster myoceanbase, trans stat result is empty. 2025-09-02 13:36:20.916 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,ec6e8563714f3caf,5ae8b20cd0a7f301] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=ec6e8563714f3caf, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:36:20.916 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,58300c47c9e0d674,214f156a8a931995] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=58300c47c9e0d674, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:36:20.916 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,0b92e3fe8ba96723,acea9c526c47b945] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=0b92e3fe8ba96723, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:36:20.917 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,58300c47c9e0d674,214f156a8a931995] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=58300c47c9e0d674, duration=4 ms] 2025-09-02 13:36:20.918 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,ec6e8563714f3caf,5ae8b20cd0a7f301] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=ec6e8563714f3caf, duration=5 ms] 2025-09-02 13:36:20.920 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,0b92e3fe8ba96723,acea9c526c47b945] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=0b92e3fe8ba96723, duration=6 ms] 2025-09-02 13:36:20.920 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,5bdef7c664f21b45,e48481a70b3a55f8] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=5bdef7c664f21b45, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:36:20.920 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,5bdef7c664f21b45,e48481a70b3a55f8] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=5bdef7c664f21b45, duration=7 ms] 2025-09-02 13:36:22.323 INFO 13484 --- [prometheus-minute-schedule-1,d653041cb2442fc5,51dfeb2a4472fc7f] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:22.328 INFO 13484 --- [prometheus-minute-schedule-1,d653041cb2442fc5,51dfeb2a4472fc7f] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:22.424 INFO 13484 --- [ocp-async-5,8ff42eccd024ceeb,b341bd81dec9d7fa] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:22.626 INFO 13484 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : Starting PLAN_CHANGE_DETECTOR, Interval{start=2025-09-02T13:32:22.002, end=2025-09-02T13:34:22.622} 2025-09-02 13:36:22.632 INFO 13484 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR : for [Pair(left=Tenant(id=1, name=sys, clusterName=myoceanbase, obClusterId=1755130302, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-29T16:03:19+08:00, primaryZone=RANDOM, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.3, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Pair(left=Tenant(id=2, name=woqu, clusterName=myoceanbase, obClusterId=1755130302, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-29T16:03:19+08:00, primaryZone=RANDOM, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.3, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null)] 2025-09-02 13:36:22.633 INFO 13484 --- [pool-16-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-09-02 13:36:22.640 INFO 13484 --- [pool-16-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 214 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=myoceanbase, obClusterId=1755130302, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-29T16:03:19+08:00, primaryZone=RANDOM, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.3, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-09-02T13:32:22.002, end=2025-09-02T13:34:22.622},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-09-02 13:36:22.686 INFO 13484 --- [ocp-schedules-20,173094f7b6568135,b875c7aa76d00fbb] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:36:22.702 INFO 13484 --- [ocp-schedules-20,173094f7b6568135,b875c7aa76d00fbb] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:36:22.894 INFO 13484 --- [pool-16-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=1, name=sys, clusterName=myoceanbase, obClusterId=1755130302, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-29T16:03:19+08:00, primaryZone=RANDOM, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.3, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null) 2025-09-02 13:36:22.895 INFO 13484 --- [pool-16-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-09-02 13:36:22.949 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:22.949 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:22.964 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704982964000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, sqlId=549981E00A3D95F03DD5A9EF6044AA20, obServerId=null, obDbId=500005, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=1, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:22.964Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=1, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=AC 2025-09-02 13:36:22.965 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704982965000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, sqlId=59C799237A82E97A5763AD0856E4F723, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=1, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:22.965Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=1, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=AC 2025-09-02 13:36:22.965 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704982965000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, sqlId=468F773521ACD3F81205B6EB7B6E4174, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=1, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:22.965Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=1, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=AC 2025-09-02 13:36:22.970 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 0 2025-09-02 13:36:22.970 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:22.982 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704982982000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, sqlId=857200A644C153C91EAA59B021904772, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=1, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:22.982Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=1, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=AC 2025-09-02 13:36:22.987 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 808 2025-09-02 13:36:22.987 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:22.989 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 12 2025-09-02 13:36:22.989 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:22.992 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:22.992 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, obServerId=null, obDbId=500005, sqlId=549981E00A3D95F03DD5A9EF6044AA20, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:22.994 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 697 2025-09-02 13:36:22.994 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:22.997 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:22.997 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.001 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.001 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.001 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, obServerId=null, obDbId=201001, sqlId=857200A644C153C91EAA59B021904772, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.001 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, obServerId=null, obDbId=201001, sqlId=468F773521ACD3F81205B6EB7B6E4174, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.006 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.006 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, obServerId=null, obDbId=201001, sqlId=59C799237A82E97A5763AD0856E4F723, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.006 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.007 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.009 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.009 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, obServerId=null, obDbId=500005, sqlId=549981E00A3D95F03DD5A9EF6044AA20, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.011 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.011 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.013 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records,use plan union hash 2025-09-02 13:36:23.013 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 0 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:22.964, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=1, sqlId=549981E00A3D95F03DD5A9EF6044AA20, serverId=null, dbName=ocs, obDbId=500005, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.020 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.020 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, obServerId=null, obDbId=201001, sqlId=468F773521ACD3F81205B6EB7B6E4174, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.025 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.025 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, obServerId=null, obDbId=201001, sqlId=59C799237A82E97A5763AD0856E4F723, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.026 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-09-02 13:36:23.034 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-09-02 13:36:23.048 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:22.965, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=1, sqlId=468F773521ACD3F81205B6EB7B6E4174, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.050 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.050 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.052 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:22.965, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=1, sqlId=59C799237A82E97A5763AD0856E4F723, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.062 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.062 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1, obServerId=null, obDbId=201001, sqlId=857200A644C153C91EAA59B021904772, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.071 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-09-02 13:36:23.081 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:22.982, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=1, sqlId=857200A644C153C91EAA59B021904772, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.082 INFO 13484 --- [pool-16-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2025-09-02 13:36:23.089 INFO 13484 --- [pool-16-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 29 diagnose targets Pair(left=Tenant(id=2, name=woqu, clusterName=myoceanbase, obClusterId=1755130302, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-29T16:03:19+08:00, primaryZone=RANDOM, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.3, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null), Interval{start=2025-09-02T13:32:22.002, end=2025-09-02T13:34:22.622},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=2000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2025-09-02 13:36:23.124 INFO 13484 --- [pool-16-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=2, name=woqu, clusterName=myoceanbase, obClusterId=1755130302, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2025-08-29T16:03:19+08:00, primaryZone=RANDOM, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.3.5.3, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, serviceNameInfo=null, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null, tags=null, loadType=null), right=null) 2025-09-02 13:36:23.124 INFO 13484 --- [pool-16-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2025-09-02 13:36:23.211 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983211000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=0F5E741E7D584B49A1A1764B314E35A3, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.211Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.212 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983212000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=C43A04459600CA43A1E4011B6DA3BBC3, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.212Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.214 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983214000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=27FEF4CF16944CBCA5B8D72AA1ECFEC0, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.214Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.217 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983217000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=CEA8D980BD2CA9A1EB289BBEACC62FD9, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.217Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.218 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983218000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=8054BA76DCD7119AA459CFB8D2371719, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.218Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.220 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983220000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=19C0CD94282664BDBE26B3E278BF6311, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.220Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.226 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983226000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=C5A5CC2B0182DD454136EC4EB1F4B645, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.226Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.233 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 263 2025-09-02 13:36:23.233 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.234 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 370 2025-09-02 13:36:23.234 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.234 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983234000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=25AE73D3BD014BC912025B2BEFAC963E, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.234Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.236 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 145 2025-09-02 13:36:23.236 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.237 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 145 2025-09-02 13:36:23.237 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.240 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 265 2025-09-02 13:36:23.240 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.241 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 268 2025-09-02 13:36:23.241 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.246 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 355 2025-09-02 13:36:23.246 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.251 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.251 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=27FEF4CF16944CBCA5B8D72AA1ECFEC0, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.253 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.253 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=8054BA76DCD7119AA459CFB8D2371719, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.253 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.253 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=0F5E741E7D584B49A1A1764B314E35A3, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.259 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.259 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.260 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 341 2025-09-02 13:36:23.260 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.262 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.262 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=C5A5CC2B0182DD454136EC4EB1F4B645, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.263 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.263 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=19C0CD94282664BDBE26B3E278BF6311, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.263 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.263 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=C43A04459600CA43A1E4011B6DA3BBC3, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.264 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.265 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.265 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.265 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.266 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.266 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=CEA8D980BD2CA9A1EB289BBEACC62FD9, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.271 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.271 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.272 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.272 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.273 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.273 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.275 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.275 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.277 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.277 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=27FEF4CF16944CBCA5B8D72AA1ECFEC0, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.281 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.281 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=25AE73D3BD014BC912025B2BEFAC963E, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.282 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.282 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=0F5E741E7D584B49A1A1764B314E35A3, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.284 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.284 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=8054BA76DCD7119AA459CFB8D2371719, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.288 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.288 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=C5A5CC2B0182DD454136EC4EB1F4B645, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.288 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.288 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.289 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-09-02 13:36:23.289 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.289 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=CEA8D980BD2CA9A1EB289BBEACC62FD9, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.290 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.290 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=19C0CD94282664BDBE26B3E278BF6311, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.293 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-09-02 13:36:23.294 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-09-02 13:36:23.295 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-09-02 13:36:23.296 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.296 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=C43A04459600CA43A1E4011B6DA3BBC3, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.297 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-09-02 13:36:23.297 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 5 records,use plan union hash 2025-09-02 13:36:23.301 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-09-02 13:36:23.305 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.211, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=0F5E741E7D584B49A1A1764B314E35A3, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.306 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.306 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=25AE73D3BD014BC912025B2BEFAC963E, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.308 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.214, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=27FEF4CF16944CBCA5B8D72AA1ECFEC0, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.308 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.218, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=8054BA76DCD7119AA459CFB8D2371719, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.312 INFO 13484 --- [plan_change_detector1,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.22, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=19C0CD94282664BDBE26B3E278BF6311, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.312 INFO 13484 --- [plan_change_detector4,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.212, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=C43A04459600CA43A1E4011B6DA3BBC3, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.313 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-09-02 13:36:23.322 INFO 13484 --- [plan_change_detector5,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.234, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=25AE73D3BD014BC912025B2BEFAC963E, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.332 INFO 13484 --- [plan_change_detector2,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 3 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.217, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=CEA8D980BD2CA9A1EB289BBEACC62FD9, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.348 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983348000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=DF4740ADC20FC47006939E33DB0F63C5, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.348Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.352 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983352000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=ACA698F4FDF258CA9E23241DBEE7AEE0, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.352Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.356 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1756704983356000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, sqlId=461F72C961338F5C08DD3D2640A4FAB3, obServerId=null, obDbId=503867, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1755130302, clusterName=myoceanbase, tenantId=2, obTenantId=1002, tenantName=woqu, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.3.5.3, startTime=2025-09-01T05:36:23.356Z, endTime=2025-09-02T05:34:22.622Z, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=172.17.172.84:2882, 2=172.17.172.85:2882, 3=172.17.172.86:2882}, serverIpMap={172.17.172.84=1, 172.17.172.85=2, 172.17.172.86=3}, cluster=ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING}, tenant=ObTenantEntity{id=2, name=woqu, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, inn 2025-09-02 13:36:23.359 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 229 2025-09-02 13:36:23.359 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.366 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 350 2025-09-02 13:36:23.366 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.367 INFO 13484 --- [plan_change_detector6,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 5 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.226, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=C5A5CC2B0182DD454136EC4EB1F4B645, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.371 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.371 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=DF4740ADC20FC47006939E33DB0F63C5, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.376 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.376 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.380 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.380 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=ACA698F4FDF258CA9E23241DBEE7AEE0, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.382 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 786 2025-09-02 13:36:23.382 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2025-09-02 13:36:23.387 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.387 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.390 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.390 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=DF4740ADC20FC47006939E33DB0F63C5, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.393 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.393 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=461F72C961338F5C08DD3D2640A4FAB3, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.394 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2025-09-02 13:36:23.398 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.398 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=ACA698F4FDF258CA9E23241DBEE7AEE0, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.399 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2025-09-02 13:36:23.399 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2025-09-02 13:36:23.404 INFO 13484 --- [plan_change_detector8,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.348, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=DF4740ADC20FC47006939E33DB0F63C5, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.408 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 5 records,use plan union hash 2025-09-02 13:36:23.415 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2025-09-02 13:36:23.415 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1756684800000000, endTimeUs=1756791262622000, obClusterId=1755130302, clusterName=myoceanbase, obTenantId=1002, obServerId=null, obDbId=503867, sqlId=461F72C961338F5C08DD3D2640A4FAB3, sqlIdList=null, dynamicSql=false) 2025-09-02 13:36:23.420 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2025-09-02 13:36:23.440 INFO 13484 --- [plan_change_detector7,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.356, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=461F72C961338F5C08DD3D2640A4FAB3, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.458 INFO 13484 --- [plan_change_detector3,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 5 plans with param:QueryTopPlanParam(interval=Interval{start=2025-09-01T13:36:23.352, end=2025-09-02T13:34:22.622}, clusterId=1, tenantId=2, sqlId=ACA698F4FDF258CA9E23241DBEE7AEE0, serverId=null, dbName=cprint, obDbId=503867, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2025-09-02 13:36:23.458 INFO 13484 --- [pool-16-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR, Found out 0 dubious sqls 2025-09-02 13:36:23.740 INFO 13484 --- [ocp-schedules-18,f143cbc2c1568587,3c50262eaeed85b4] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-09-02 13:36:23.748 INFO 13484 --- [ocp-schedules-18,f143cbc2c1568587,3c50262eaeed85b4] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:36:23.748 INFO 13484 --- [ocp-schedules-18,f143cbc2c1568587,3c50262eaeed85b4] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:36:24.424 INFO 13484 --- [ocp-async-2,a9368218c420e61a,31ba6e66904685fb] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:25.323 INFO 13484 --- [prometheus-minute-schedule-1,5bf1013b7aa9f3c5,55a6e2525d4e0adf] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:25.327 INFO 13484 --- [prometheus-minute-schedule-1,5bf1013b7aa9f3c5,55a6e2525d4e0adf] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:25.949 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:25.950 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:26.036 INFO 13484 --- [ocp-schedules-14,05c057b04510d4ed,4c577852fc762862] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:36:26.050 WARN 13484 --- [scanner-12209047-1,05c057b04510d4ed,9dc21d1b0ebae2cc] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:36:26.424 INFO 13484 --- [ocp-async-4,f2b25ec6d3a64114,2a62f533f044317c] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:26.539 INFO 13484 --- [ocp-schedules-26,13ebca2d77854d11,aa4fde0ed7bfbce2] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-09-02 13:36:26.552 INFO 13484 --- [ocp-schedules-29,5dae51c6333fea24,152a07c1e0346a73] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:36:26.552 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=172.19.62.154, originalStatus=ONLINE 2025-09-02 13:36:26.552 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=172.19.62.156, originalStatus=ONLINE 2025-09-02 13:36:26.553 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=172.17.172.86, originalStatus=ONLINE 2025-09-02 13:36:26.553 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=172.17.172.85, originalStatus=ONLINE 2025-09-02 13:36:26.553 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=172.17.172.84, originalStatus=ONLINE 2025-09-02 13:36:26.556 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:36:26.556 INFO 13484 --- [ocp-schedules-18,7115ecfa27a06463,ab06272f2f0dc692] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:36:26.556 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:36:26.557 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:36:26.557 INFO 13484 --- [host-check-agent-status1,7115ecfa27a06463,26bc0d3e9b54f172] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=172.19.62.154, originalStatus=ONLINE 2025-09-02 13:36:26.557 INFO 13484 --- [host-check-agent-status2,7115ecfa27a06463,0728da450b427398] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=172.19.62.156, originalStatus=ONLINE 2025-09-02 13:36:26.557 INFO 13484 --- [host-check-agent-status3,7115ecfa27a06463,021dc5be067b3fcf] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=172.17.172.84, originalStatus=ONLINE 2025-09-02 13:36:26.557 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:36:26.557 INFO 13484 --- [host-check-agent-status4,7115ecfa27a06463,ad25384790444c76] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=172.17.172.86, originalStatus=ONLINE 2025-09-02 13:36:26.557 INFO 13484 --- [host-check-agent-status5,7115ecfa27a06463,39d1b1e10e6052ce] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=172.17.172.85, originalStatus=ONLINE 2025-09-02 13:36:26.558 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:36:26.558 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:36:26.558 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:36:26.558 INFO 13484 --- [ocp-schedules-22,7a085b67ff53a92e,770856b5f95a0dac] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-09-02 13:36:26.558 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:36:26.562 INFO 13484 --- [ocp-schedules-22,7a085b67ff53a92e,770856b5f95a0dac] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:26.562 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:36:26.562 INFO 13484 --- [host-check-agent-status1,7115ecfa27a06463,26bc0d3e9b54f172] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:36:26.563 INFO 13484 --- [host-check-agent-status4,7115ecfa27a06463,ad25384790444c76] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:36:26.564 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:36:26.564 INFO 13484 --- [ocp-schedules-9,a81aa33018c31dcf,5ce6852b53f59c73] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:36:26.564 INFO 13484 --- [host-check-clockdiff1,a81aa33018c31dcf,d4db38827527ed55] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.19.62.154], timeoutMillis=20000 2025-09-02 13:36:26.565 INFO 13484 --- [host-check-clockdiff2,a81aa33018c31dcf,f2ddae1a63a4ddb2] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.19.62.156], timeoutMillis=20000 2025-09-02 13:36:26.565 INFO 13484 --- [host-check-agent-status1,7115ecfa27a06463,26bc0d3e9b54f172] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:36:26.565 INFO 13484 --- [ocp-schedules-3,baf16136c2ff712b,5adc50dadc2acaa8] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-09-02 13:36:26.565 INFO 13484 --- [ocp-schedules-28,9bf83ae78b9837a4,63f5b7e62ae5bcd4] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-09-02 13:36:26.566 INFO 13484 --- [ocp-schedules-28,9bf83ae78b9837a4,63f5b7e62ae5bcd4] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:26.566 INFO 13484 --- [host-check-agent-status3,7115ecfa27a06463,021dc5be067b3fcf] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:36:26.566 INFO 13484 --- [ocp-schedules-3,baf16136c2ff712b,5adc50dadc2acaa8] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:26.567 INFO 13484 --- [host-check-agent-status4,7115ecfa27a06463,ad25384790444c76] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:36:26.568 INFO 13484 --- [host-check-clockdiff4,a81aa33018c31dcf,6f1bfce840da95bf] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.86], timeoutMillis=20000 2025-09-02 13:36:26.568 INFO 13484 --- [host-check-clockdiff3,a81aa33018c31dcf,f8e44de6c3352f31] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.84], timeoutMillis=20000 2025-09-02 13:36:26.565 INFO 13484 --- [host-check-agent-status2,7115ecfa27a06463,0728da450b427398] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:36:26.569 INFO 13484 --- [host-check-clockdiff5,a81aa33018c31dcf,695d05c891678a8f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.85], timeoutMillis=20000 2025-09-02 13:36:26.570 INFO 13484 --- [host-check-agent-status5,7115ecfa27a06463,39d1b1e10e6052ce] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:36:26.570 INFO 13484 --- [host-check-agent-status3,7115ecfa27a06463,021dc5be067b3fcf] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:36:26.570 INFO 13484 --- [host-check-agent-status2,7115ecfa27a06463,0728da450b427398] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:36:26.571 INFO 13484 --- [host-check-agent-status5,7115ecfa27a06463,39d1b1e10e6052ce] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:36:26.594 INFO 13484 --- [host-check-clockdiff5,a81aa33018c31dcf,695d05c891678a8f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791386 -1 -1 , error= 2025-09-02 13:36:26.599 INFO 13484 --- [ocp-schedules-23,0d7f412071debc25,79117ede8c1f7dd8] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-09-02 13:36:26.603 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62888/api/v1/time, params:null 2025-09-02 13:36:26.605 INFO 13484 --- [ocp-schedules-3,baf16136c2ff712b,5adc50dadc2acaa8] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=1, successCount=1 2025-09-02 13:36:26.605 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=172.19.62.154, timestamp=2025-09-02T13:36:26.605404609+08:00 2025-09-02 13:36:26.607 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-09-02T13:36:26.606+08:00 2025-09-02 13:36:26.622 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-09-02T13:36:26.606+08:00 and status AVAILABLE 2025-09-02 13:36:26.624 INFO 13484 --- [host-check-clockdiff1,a81aa33018c31dcf,d4db38827527ed55] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791386 0 0 , error= 2025-09-02 13:36:26.624 INFO 13484 --- [host-check-clockdiff3,a81aa33018c31dcf,f8e44de6c3352f31] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791386 0 0 , error= 2025-09-02 13:36:26.625 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62888/api/v1/time, params:null 2025-09-02 13:36:26.632 INFO 13484 --- [host-check-clockdiff4,a81aa33018c31dcf,6f1bfce840da95bf] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791386 -1 -1 , error= 2025-09-02 13:36:26.634 INFO 13484 --- [host-check-clockdiff2,a81aa33018c31dcf,f2ddae1a63a4ddb2] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791386 0 0 , error= 2025-09-02 13:36:26.635 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=172.17.172.85, timestamp=2025-09-02T13:36:26.634455372+08:00 2025-09-02 13:36:26.636 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-09-02T13:36:26.635+08:00 2025-09-02 13:36:26.636 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-09-02 13:36:26.637 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62888/api/v1/time, params:null 2025-09-02 13:36:26.637 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62888/api/v1/time, params:null 2025-09-02 13:36:26.638 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-09-02T13:36:26.635+08:00 and status AVAILABLE 2025-09-02 13:36:26.639 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62888/api/v1/time, params:null 2025-09-02 13:36:26.639 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=172.19.62.156, timestamp=2025-09-02T13:36:26.638865546+08:00 2025-09-02 13:36:26.640 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=172.17.172.84, timestamp=2025-09-02T13:36:26.639281933+08:00 2025-09-02 13:36:26.640 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-09-02T13:36:26.640+08:00 2025-09-02 13:36:26.641 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-09-02T13:36:26.639+08:00 2025-09-02 13:36:26.641 INFO 13484 --- [host-check-agent-status4,7115ecfa27a06463,ad25384790444c76] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62888/api/v1/agent/status, params:null 2025-09-02 13:36:26.641 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=172.17.172.86, timestamp=2025-09-02T13:36:26.640654064+08:00 2025-09-02 13:36:26.641 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-09-02T13:36:26.641+08:00 2025-09-02 13:36:26.642 INFO 13484 --- [host-check-agent-status2,7115ecfa27a06463,0728da450b427398] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62888/api/v1/agent/status, params:null 2025-09-02 13:36:26.642 INFO 13484 --- [host-check-agent-status1,7115ecfa27a06463,26bc0d3e9b54f172] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62888/api/v1/agent/status, params:null 2025-09-02 13:36:26.643 INFO 13484 --- [host-check-agent-status5,7115ecfa27a06463,39d1b1e10e6052ce] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62888/api/v1/agent/status, params:null 2025-09-02 13:36:26.643 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-09-02 13:36:26.643 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-09-02T13:36:26.640+08:00 and status AVAILABLE 2025-09-02 13:36:26.646 INFO 13484 --- [host-check-agent-status1,7115ecfa27a06463,26bc0d3e9b54f172] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.19.62.154 2025-09-02 13:36:26.646 INFO 13484 --- [host-check-agent-status1,7115ecfa27a06463,26bc0d3e9b54f172] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.19.62.154 2025-09-02 13:36:26.646 INFO 13484 --- [host-check-agent-status1,7115ecfa27a06463,26bc0d3e9b54f172] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.19.62.154 2025-09-02 13:36:26.646 INFO 13484 --- [host-check-agent-status5,7115ecfa27a06463,39d1b1e10e6052ce] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.85 2025-09-02 13:36:26.646 INFO 13484 --- [host-check-agent-status5,7115ecfa27a06463,39d1b1e10e6052ce] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.85 2025-09-02 13:36:26.646 INFO 13484 --- [host-check-agent-status5,7115ecfa27a06463,39d1b1e10e6052ce] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.85 2025-09-02 13:36:26.647 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-09-02T13:36:26.641+08:00 and status AVAILABLE 2025-09-02 13:36:26.647 INFO 13484 --- [host-check-agent-status4,7115ecfa27a06463,ad25384790444c76] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.86 2025-09-02 13:36:26.647 INFO 13484 --- [host-check-agent-status4,7115ecfa27a06463,ad25384790444c76] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.86 2025-09-02 13:36:26.647 INFO 13484 --- [host-check-agent-status4,7115ecfa27a06463,ad25384790444c76] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.86 2025-09-02 13:36:26.647 INFO 13484 --- [host-check-agent-status3,7115ecfa27a06463,021dc5be067b3fcf] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62888/api/v1/agent/status, params:null 2025-09-02 13:36:26.647 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-09-02T13:36:26.639+08:00 and status AVAILABLE 2025-09-02 13:36:26.648 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:36:26.648 INFO 13484 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:26.649 INFO 13484 --- [host-check-agent-status2,7115ecfa27a06463,0728da450b427398] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.19.62.156 2025-09-02 13:36:26.649 INFO 13484 --- [host-check-agent-status2,7115ecfa27a06463,0728da450b427398] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.19.62.156 2025-09-02 13:36:26.649 INFO 13484 --- [host-check-agent-status2,7115ecfa27a06463,0728da450b427398] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.19.62.156 2025-09-02 13:36:26.649 INFO 13484 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-09-02 13:36:26.650 INFO 13484 --- [host-check-agent-status3,7115ecfa27a06463,021dc5be067b3fcf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.84 2025-09-02 13:36:26.650 INFO 13484 --- [host-check-agent-status3,7115ecfa27a06463,021dc5be067b3fcf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.84 2025-09-02 13:36:26.650 INFO 13484 --- [host-check-agent-status3,7115ecfa27a06463,021dc5be067b3fcf] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.84 2025-09-02 13:36:26.650 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:36:26.651 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-09-02 13:36:26.653 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:36:26.653 INFO 13484 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.3.5.3 2025-09-02 13:36:26.654 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-09-02 13:36:26.654 INFO 13484 --- [ocp-schedules-22,7a085b67ff53a92e,770856b5f95a0dac] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:36:26.654 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-09-02 13:36:26.654 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:36:26.660 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:36:26.662 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:36:26.664 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:36:26.664 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:36:26.665 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:36:26.666 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:36:26.667 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:26.668 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-09-02 13:36:26.669 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:36:26.675 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:36:26.687 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:36:26.688 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:36:26.689 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:36:26.703 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:26.705 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-09-02 13:36:26.705 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:26.706 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-09-02 13:36:26.706 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-09-02 13:36:26.710 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-09-02 13:36:26.714 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-09-02 13:36:26.714 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-09-02 13:36:26.718 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-09-02 13:36:26.729 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:36:26.735 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:36:26.737 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-09-02 13:36:26.738 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:36:26.745 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:26.747 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:26.748 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-09-02 13:36:26.748 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-09-02 13:36:26.751 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:26.752 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-09-02 13:36:26.754 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-09-02 13:36:26.755 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-09-02 13:36:26.757 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:36:26.761 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-09-02 13:36:26.762 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-09-02 13:36:26.762 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-09-02 13:36:26.763 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-09-02 13:36:26.767 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:36:26.771 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:26.771 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.025724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:36:26.772 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-09-02 13:36:26.774 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-09-02 13:36:26.783 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:36:26.787 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:36:26.789 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:36:26.791 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:36:26.791 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.025724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:26.791 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:36:26.804 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:26.804 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-09-02 13:36:26.804 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27.368133+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:36:26.805 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-09-02 13:36:26.812 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-09-02 13:36:26.813 INFO 13484 --- [host-check-host-status4,5dae51c6333fea24,909469c1b1f26767] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-09-02 13:36:26.814 INFO 13484 --- [host-check-host-status5,5dae51c6333fea24,0be35678ff4001a3] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-09-02 13:36:26.821 INFO 13484 --- [host-check-host-status3,5dae51c6333fea24,baee6e407cd8c960] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-09-02 13:36:26.822 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27.368133+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:26.822 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:36:26.835 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:26.835 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.373150+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:36:26.843 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-09-02 13:36:26.853 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-09-02 13:36:26.853 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.373150+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:26.868 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-09-02 13:36:26.869 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:26.870 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:26.870 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:36:26.871 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-09-02 13:36:26.875 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster myoceanbase 2025-09-02 13:36:26.876 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-09-02 13:36:26.880 INFO 13484 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 3, sqlPort: 2881 2025-09-02 13:36:26.884 INFO 13484 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:36:26.885 INFO 13484 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:36:26.893 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:36:26.904 INFO 13484 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-09-02 13:36:26.941 INFO 13484 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 3, sqlPort: 2881 got ObProcessInfoResult(pid=4794, runPath=/data1/myoceanbase/oceanbase, runUser=root, execute=/data1/myoceanbase/oceanbase/bin/observer, installPath=/data1/myoceanbase/oceanbase) 2025-09-02 13:36:26.942 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-09-02 13:36:26.951 INFO 13484 --- [host-check-host-status1,5dae51c6333fea24,a77bb8c2f8b80b9e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-09-02 13:36:26.955 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:26.956 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:26.957 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-09-02 13:36:26.968 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-09-02 13:36:26.974 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=myoceanbase, obClusterId=1755130302 2025-09-02 13:36:26.974 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=myoceanbase, obClusterId=1755130302, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"myoceanbase","ObClusterId":1755130302,"ObRegion":"myoceanbase","ObRegionId":1755130302,"ReadonlyRsList":[],"RsList":[{"address":"172.17.172.85:2882","role":"LEADER","sql_port":2881},{"address":"172.17.172.86:2882","role":"FOLLOWER","sql_port":2881},{"address":"172.17.172.84:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1756791174423000}, createTime=2025-08-29T16:02:30+08:00, creator=admin, obVersion=4.3.5.3, packageOperatingSystem=null, loadType=null, attributesJson={"DataDiskPath":"/data1/myoceanbase/oceanbase/store","DiskPathStyle":"OBD","InstallPath":"/data1/myoceanbase/oceanbase","LogDiskPath":"/data1/myoceanbase/oceanbase/store","OperatingSystemUser":"root","RunPath":"/data1/myoceanbase/oceanbase","SqlPort":2881,"SvrPort":2882}, tenantCount=2) 2025-09-02 13:36:26.974 INFO 13484 --- [ocp-schedules-28,9bf83ae78b9837a4,63f5b7e62ae5bcd4] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:36:26.978 INFO 13484 --- [host-check-host-status2,5dae51c6333fea24,9e49c74f2499fd99] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-09-02 13:36:26.985 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:36:26.984793+08:00, lastProcessTime=2025-09-02T13:36:16.985166+08:00 2025-09-02 13:36:27.138 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:36:26.984793+08:00 2025-09-02 13:36:27.913 INFO 13484 --- [ocp-schedules-30,6e3372e7a16a65a6,92557ab3ef3e9dde] c.o.o.s.common.scheduled.PerfSchedules : Collect slow sql for alarm, time=2025-09-02T13:36:27.913+08:00, clusters=[1] 2025-09-02 13:36:27.937 INFO 13484 --- [ocp-schedules-30,6e3372e7a16a65a6,92557ab3ef3e9dde] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for myoceanbase, save 7 to SqlAggregateData 2025-09-02 13:36:28.239 INFO 13484 --- [ocp-schedules-9,3e83c65e4cb6e4da,77b40ba0a3547fd4] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:36:28.323 INFO 13484 --- [prometheus-minute-schedule-1,db30055e41674679,a455762d60ce1c55] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:28.327 INFO 13484 --- [prometheus-minute-schedule-1,db30055e41674679,a455762d60ce1c55] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:28.424 INFO 13484 --- [ocp-async-3,30d141047f22325f,9ab006f6bbc819fa] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:28.950 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:28.951 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:29.093 INFO 13484 --- [ocp-async-10,fc34ad832fa370ef,9084725f2bb0b621] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:36:29.544 INFO 13484 --- [ocp-schedules-22,12da0b22bdbb7087,ee3071308770777a] c.o.o.s.c.scheduled.TenantSchedules : Sync all tenant info schedule. 2025-09-02 13:36:29.546 INFO 13484 --- [ocp-schedules-22,12da0b22bdbb7087,ee3071308770777a] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:29.595 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-09-02 13:36:29.595 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.598 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) ocp_query */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-09-02 13:36:29.616 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.617 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-09-02 13:36:29.645 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.646 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-09-02 13:36:29.678 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.678 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1] 2025-09-02 13:36:29.682 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.683 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2025-09-02 13:36:29.709 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.709 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2025-09-02 13:36:29.736 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.737 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1002] 2025-09-02 13:36:29.755 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.756 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2025-09-02 13:36:29.758 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.759 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-09-02 13:36:29.762 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.763 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-09-02 13:36:29.765 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.766 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2025-09-02 13:36:29.768 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.769 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-09-02 13:36:29.772 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.773 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-09-02 13:36:29.775 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2025-09-02 13:36:29.783 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=172.19.62.162 2025-09-02 13:36:29.809 INFO 13484 --- [Sync all tenant information1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:29.811 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:29.811 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE name = ?, args: [ob_tcp_invited_nodes] 2025-09-02 13:36:29.826 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,172.19.62.162, userWhitelist=% 2025-09-02 13:36:29.826 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2025-09-02 13:36:29.826 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,172.19.62.162, userWhitelist=% 2025-09-02 13:36:29.826 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2025-09-02 13:36:29.828 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2025-09-02 13:36:29.828 INFO 13484 --- [Sync all tenant information1,,] .o.s.t.b.t.SyncAllTenantInfoShardingTask : [OCP606] sync tenants success, clusterId=1, clusterName=myoceanbase, obClusterId=1755130302 2025-09-02 13:36:29.829 INFO 13484 --- [ocp-schedules-22,12da0b22bdbb7087,ee3071308770777a] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all tenant information executed completed, allCount=1, successCount=1 2025-09-02 13:36:30.425 INFO 13484 --- [ocp-async-2,e8a5b00ab300998a,422d6dd8db3d2aa5] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:30.933 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,3b9be09ce4cc1234,b36a69e6b3a7b076] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=3b9be09ce4cc1234, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:36:30.933 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,e7ae9deca7420661,c331ea6ecec87ced] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=e7ae9deca7420661, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:36:30.933 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,90081ce25641aa80,025da47f25488d7f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=90081ce25641aa80, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:36:30.934 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,e7ae9deca7420661,c331ea6ecec87ced] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=e7ae9deca7420661, duration=5 ms] 2025-09-02 13:36:30.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,3e38a71af1b565e4,21ff8c3c3588c86d] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=3e38a71af1b565e4, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:36:30.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,3b9be09ce4cc1234,b36a69e6b3a7b076] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=3b9be09ce4cc1234, duration=6 ms] 2025-09-02 13:36:30.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,3e38a71af1b565e4,21ff8c3c3588c86d] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=3e38a71af1b565e4, duration=6 ms] 2025-09-02 13:36:30.936 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,90081ce25641aa80,025da47f25488d7f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=90081ce25641aa80, duration=7 ms] 2025-09-02 13:36:31.323 INFO 13484 --- [prometheus-minute-schedule-1,20851ee073fbef45,df143f67f0da2663] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:31.328 INFO 13484 --- [prometheus-minute-schedule-1,20851ee073fbef45,df143f67f0da2663] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:31.951 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:31.952 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:32.425 INFO 13484 --- [ocp-async-4,9bf63660effeeb4f,2c5816ea97ff8bc6] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:32.708 INFO 13484 --- [ocp-schedules-13,5114148b33bc0e81,d77a6a92102f34b4] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:36:32.711 INFO 13484 --- [ocp-schedules-13,5114148b33bc0e81,d77a6a92102f34b4] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:36:34.323 INFO 13484 --- [prometheus-minute-schedule-1,f807750b747f0748,fe3084cbec032904] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:34.328 INFO 13484 --- [prometheus-minute-schedule-1,f807750b747f0748,fe3084cbec032904] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:34.425 INFO 13484 --- [ocp-async-5,380f3563cdb414eb,0e50bca6b4dee7d2] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:34.920 INFO 13484 --- [ocp-schedules-6,5f596b48385f1183,dc3f9dde705e2c2c] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-09-02 13:36:34.952 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:34.952 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:36.117 INFO 13484 --- [ocp-schedules-30,54aaeeca3fbf8198,c11a360042da3e73] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:36:36.141 WARN 13484 --- [scanner-12209207-1,54aaeeca3fbf8198,867331fceb66d4cd] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:36:36.157 INFO 13484 --- [alarm-detect-1,54aaeeca3fbf8198,e5be4044d9586952] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 3016171, data is empty. 2025-09-02 13:36:36.166 INFO 13484 --- [alarm-detect-9,54aaeeca3fbf8198,ad594dff018aacab] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 3016171, data is empty. 2025-09-02 13:36:36.166 INFO 13484 --- [alarm-detect-11,54aaeeca3fbf8198,2df59e436fccd475] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 3016171, data is empty. 2025-09-02 13:36:36.217 INFO 13484 --- [alarm-detect-8,54aaeeca3fbf8198,848513bd66b14ee2] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 3016171, data is empty. 2025-09-02 13:36:36.425 INFO 13484 --- [ocp-async-1,1f45cb1dffcf9f8e,76a56e2fc7c46b98] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:36.987 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:36:36.986605+08:00, lastProcessTime=2025-09-02T13:36:26.984793+08:00 2025-09-02 13:36:37.141 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:36:36.986605+08:00 2025-09-02 13:36:37.323 INFO 13484 --- [prometheus-minute-schedule-1,c2bdddad1b5d1239,6fe66474f1bd3e7e] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:37.328 INFO 13484 --- [prometheus-minute-schedule-1,c2bdddad1b5d1239,6fe66474f1bd3e7e] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:37.953 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:37.953 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:38.244 INFO 13484 --- [ocp-schedules-23,19f4e9b951ecac49,cf16164314c3b46c] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:36:38.425 INFO 13484 --- [ocp-async-8,3adc636991ea4bf3,82a79493878a4688] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:39.093 INFO 13484 --- [ocp-async-6,9b174f597f659b60,8b3f0a56cb87075a] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:36:40.004 INFO 13484 --- [ocp-schedules-12,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=2 2025-09-02 13:36:40.005 INFO 13484 --- [ocp-schedules-24,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=2 2025-09-02 13:36:40.019 INFO 13484 --- [Check obproxy2,e30b27ee97c90d32,57fc05ba25050440] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:36:40.020 INFO 13484 --- [Check obproxy2,e30b27ee97c90d32,57fc05ba25050440] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:36:40.020 INFO 13484 --- [Check obproxy1,737ef8bd07d08c07,a2a7e8cbe42b2be9] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:36:40.020 INFO 13484 --- [Check obproxy1,737ef8bd07d08c07,a2a7e8cbe42b2be9] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:36:40.024 INFO 13484 --- [Collect obproxy2,e4f41401fc88be4b,64f045effb6815e8] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:36:40.024 INFO 13484 --- [Collect obproxy2,e4f41401fc88be4b,64f045effb6815e8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:36:40.030 INFO 13484 --- [Collect obproxy1,f2ef6bd06c8df470,915ef652803a1572] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:36:40.030 INFO 13484 --- [Collect obproxy1,f2ef6bd06c8df470,915ef652803a1572] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:36:40.042 INFO 13484 --- [Check obproxy1,737ef8bd07d08c07,a2a7e8cbe42b2be9] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:36:40.043 INFO 13484 --- [Check obproxy2,e30b27ee97c90d32,57fc05ba25050440] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:36:40.045 INFO 13484 --- [Check obproxy2,e30b27ee97c90d32,57fc05ba25050440] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-09-02 13:36:40.045 INFO 13484 --- [Check obproxy2,e30b27ee97c90d32,57fc05ba25050440] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 5, need to sync to meta: false 2025-09-02 13:36:40.046 INFO 13484 --- [Check obproxy1,737ef8bd07d08c07,a2a7e8cbe42b2be9] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-09-02 13:36:40.046 INFO 13484 --- [Check obproxy1,737ef8bd07d08c07,a2a7e8cbe42b2be9] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 6, need to sync to meta: false 2025-09-02 13:36:40.051 INFO 13484 --- [Collect obproxy2,e4f41401fc88be4b,64f045effb6815e8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:36:40.054 INFO 13484 --- [Collect obproxy2,e4f41401fc88be4b,64f045effb6815e8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:36:40.055 INFO 13484 --- [Collect obproxy1,f2ef6bd06c8df470,915ef652803a1572] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:36:40.057 INFO 13484 --- [Collect obproxy2,e4f41401fc88be4b,64f045effb6815e8] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:36:40.057 INFO 13484 --- [Collect obproxy1,f2ef6bd06c8df470,915ef652803a1572] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:36:40.060 INFO 13484 --- [Collect obproxy1,f2ef6bd06c8df470,915ef652803a1572] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:36:40.323 INFO 13484 --- [prometheus-minute-schedule-1,bed6351c7e6f58f1,adf80435585fe68d] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:40.328 INFO 13484 --- [prometheus-minute-schedule-1,bed6351c7e6f58f1,adf80435585fe68d] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:40.426 INFO 13484 --- [ocp-async-5,12234258c3203e17,0ceec4eeeea73ca2] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:40.930 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,f1d755dac94ae7f1,af9f5703eed14663] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=f1d755dac94ae7f1, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:36:40.930 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,2dcdaf4b96212418,159f4f1182351bc4] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=2dcdaf4b96212418, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:36:40.930 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,215093514d91a3db,f8333c722b68e47e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=215093514d91a3db, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:36:40.930 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,63edd247c0bf2cbb,cf2fef551d04c538] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=63edd247c0bf2cbb, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:36:40.930 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,215093514d91a3db,f8333c722b68e47e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=215093514d91a3db, duration=4 ms] 2025-09-02 13:36:40.931 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,2dcdaf4b96212418,159f4f1182351bc4] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=2dcdaf4b96212418, duration=5 ms] 2025-09-02 13:36:40.931 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,63edd247c0bf2cbb,cf2fef551d04c538] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=63edd247c0bf2cbb, duration=5 ms] 2025-09-02 13:36:40.932 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,f1d755dac94ae7f1,af9f5703eed14663] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=f1d755dac94ae7f1, duration=6 ms] 2025-09-02 13:36:40.953 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:40.954 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:41.323 INFO 13484 --- [prometheus-cache-monitor1,8280f8f79283e389,d70e4eb7ef5d3b78] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":10712,"minuteWriteQueue":0,"minuteReadCacheSize":4820,"secondWriteCacheSize":2678,"seriesIdKeySize":5749,"secondWriteQueue":0,"minuteWriteCacheSize":4806} 2025-09-02 13:36:41.323 INFO 13484 --- [prometheus-cache-monitor1,8280f8f79283e389,d70e4eb7ef5d3b78] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{ob_sysstat_40012=15, ob_server_resource_memory_assigned_bytes=10, ob_sysstat_40011=15, ob_sysstat_40010=15, ob_unit_num=10, ob_database_disk_required_size=29, go_memstats_mspan_sys_bytes=16, ob_sysstat_30213=15, go_memstats_stack_sys_bytes=16, ob_sysstat_30212=15, ob_sysstat_30217=15, ob_sysstat_30216=15, ob_tenant_request_queue_total_size=16, jvm_gc_memory_allocated_bytes_total=2, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=10, ob_sysstat_40019=15, ob_sysstat_40018=15, processor_process_metrics_total=20, jvm_gc_max_data_size_bytes=2, ob_sysstat_40027=15, ob_sysstat_40026=15, ob_sysstat_40025=15, go_memstats_frees_total=16, host_available=15, ob_sysstat_30220=15, ob_sysstat_30222=15, ob_sysstat_30221=15, mysql_output_write_sql_failed_count=19, ob_database_disk_data_size=28, ob_tenant_memstore_freeze_cnt=12, ob_tenant_unit_min_memory=21, ob_sysstat_40028=15, ob_sysstat_60087=15, ob_memstore_total_bytes=16, jvm_threads_states_threads=8, process_resident_memory_bytes=16, ob_cache_size_bytes=33, ob_tenant_unit_max_memory=21, input_collect_metrics_total=24, ob_sysstat_50039=15, ob_sysstat_60083=15, ob_sysstat_50038=15, node_disk_io_time_weighted_seconds_total=27, process_open_fds=16, ob_sysstat_50037=15, process_memory_rss_bytes=29, ob_sysstat_50030=15, ob_server_resource_cpu_assigned_percent=10, sql_audit_input_collect_lost_total=12, go_memstats_heap_inuse_bytes=16, ob_sysstat_40118=15, ob_sysstat_40117=15, node_memory_memfree_bytes=20, ob_sysstat_40116=15, ob_sysstat_40005=15, process_exists=39, ob_sysstat_40004=15, ob_sysstat_40003=15, ob_sysstat_40002=15, go_gc_duration_seconds=21, ob_server_resource_disk_bytes=10, ob_sysstat_40001=15, ob_sysstat_40000=15, mysql_output_write_sql_time=19, ob_sysstat_30200=15, odp_current_session=17, ob_sysstat_30201=15, ob_sysstat_30204=15, ob_sysstat_30205=15, ob_sysstat_30208=15, process_thread_count=29, ob_sysstat_30209=15, ob_sysstat_50040=15, ob_sysstat_40009=15, ob_disk_free_bytes=10, ob_sysstat_40008=15, node_filesystem_avail_bytes=49, 2025-09-02 13:36:42.426 INFO 13484 --- [ocp-async-1,e78567d0186312b2,56002f7314e17810] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:42.720 INFO 13484 --- [ocp-schedules-30,0e1f4b10967b50b3,df365e52451a8538] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:36:42.723 INFO 13484 --- [ocp-schedules-30,0e1f4b10967b50b3,df365e52451a8538] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:36:43.323 INFO 13484 --- [prometheus-minute-schedule-1,c19536353d6c554c,047039d6e0433ea2] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:43.328 INFO 13484 --- [prometheus-minute-schedule-1,c19536353d6c554c,047039d6e0433ea2] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:43.750 INFO 13484 --- [ocp-schedules-12,3c3d37603eb4c3d9,4773e6c890be2917] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-09-02 13:36:43.758 INFO 13484 --- [ocp-schedules-12,3c3d37603eb4c3d9,4773e6c890be2917] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:36:43.758 INFO 13484 --- [ocp-schedules-12,3c3d37603eb4c3d9,4773e6c890be2917] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:36:43.954 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:43.955 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:44.426 INFO 13484 --- [ocp-async-8,9f38a9725e74a6f2,d5254bfd27defca4] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:45.101 INFO 13484 --- [prometheus-second-schedule-1,93903aebe481ec6b,2ea9b8b120ff80fd] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=15, collectTaskCount=15 2025-09-02 13:36:46.228 INFO 13484 --- [ocp-schedules-24,861c140d1aee270f,d9724d12e356dd09] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:36:46.263 WARN 13484 --- [scanner-12209362-1,861c140d1aee270f,94f0f2d39da3f8d8] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:36:46.332 INFO 13484 --- [prometheus-minute-schedule-1,036242765579be31,27c48d3cea02cadf] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:46.344 INFO 13484 --- [prometheus-minute-schedule-1,036242765579be31,27c48d3cea02cadf] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:46.426 INFO 13484 --- [ocp-async-9,c65e16ebb3485920,7f6d50b7ca35e711] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:46.955 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:46.956 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:46.987 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:36:46.985894+08:00, lastProcessTime=2025-09-02T13:36:36.986605+08:00 2025-09-02 13:36:47.141 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:36:46.985894+08:00 2025-09-02 13:36:48.248 INFO 13484 --- [ocp-schedules-15,282c471b13553c64,e5cb2098c9c3f819] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:36:48.426 INFO 13484 --- [ocp-async-3,37be611c2b581c6e,4d6a6391bacee513] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:48.891 INFO 13484 --- [ocp-schedules-26,576f8cd33bff0730,9a965a0b361ed330] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster zone cgroup value. 2025-09-02 13:36:48.892 INFO 13484 --- [ocp-schedules-26,576f8cd33bff0730,9a965a0b361ed330] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:48.912 INFO 13484 --- [Check cluster cgroup value.1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:48.913 INFO 13484 --- [Check cluster cgroup value.1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [enable_cgroup] 2025-09-02 13:36:48.924 INFO 13484 --- [ocp-schedules-26,576f8cd33bff0730,9a965a0b361ed330] .o.o.s.t.b.c.AbstractShardingClusterTask : Check cluster cgroup value. executed completed, allCount=1, successCount=1 2025-09-02 13:36:49.093 INFO 13484 --- [ocp-async-5,f4745a45ff1494e4,e858b9e25b242e11] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:36:49.323 INFO 13484 --- [prometheus-minute-schedule-1,10da0365fb105b4c,0341679b9046565f] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:49.328 INFO 13484 --- [prometheus-minute-schedule-1,10da0365fb105b4c,0341679b9046565f] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:49.956 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:49.957 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:50.427 INFO 13484 --- [ocp-async-8,e0b8659af507b27d,a54eeb68e4031e97] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:50.929 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,26f0c916c8b03794,616d100a380ec9e6] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=26f0c916c8b03794, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:36:50.930 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,88de7345162d0d72,9c1d6415dedef90f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=88de7345162d0d72, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:36:50.930 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,d1d62fd3ae0b082b,9b22f503848c763c] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=d1d62fd3ae0b082b, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:36:50.930 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,88de7345162d0d72,9c1d6415dedef90f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=88de7345162d0d72, duration=4 ms] 2025-09-02 13:36:50.930 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,d1d62fd3ae0b082b,9b22f503848c763c] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=d1d62fd3ae0b082b, duration=4 ms] 2025-09-02 13:36:50.930 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,26f0c916c8b03794,616d100a380ec9e6] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=26f0c916c8b03794, duration=4 ms] 2025-09-02 13:36:50.932 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,eec6d7e69b1358e8,fc9eb0dea798ed33] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=eec6d7e69b1358e8, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:36:50.934 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,eec6d7e69b1358e8,fc9eb0dea798ed33] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=eec6d7e69b1358e8, duration=8 ms] 2025-09-02 13:36:52.323 INFO 13484 --- [prometheus-minute-schedule-1,555d0e29dc2d679d,55c16e62cfa30d11] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:52.327 INFO 13484 --- [prometheus-minute-schedule-1,555d0e29dc2d679d,55c16e62cfa30d11] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:52.427 INFO 13484 --- [ocp-async-9,945d212cba8d99c3,a00bb64418b4f41c] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:52.731 INFO 13484 --- [ocp-schedules-29,edb084e92b2be99c,b91835415171372c] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:36:52.733 INFO 13484 --- [ocp-schedules-29,edb084e92b2be99c,b91835415171372c] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:36:52.957 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:52.958 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:54.427 INFO 13484 --- [ocp-async-3,3ef41384cbc0828d,f30e98c665eace0f] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:55.323 INFO 13484 --- [prometheus-minute-schedule-1,345c740b558bcad4,19a3a880f29111c9] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:55.328 INFO 13484 --- [prometheus-minute-schedule-1,345c740b558bcad4,19a3a880f29111c9] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:55.958 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:55.959 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:56.337 INFO 13484 --- [ocp-schedules-5,26fd4d58431bb166,e19c4741bb41c792] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:36:56.366 INFO 13484 --- [alarm-detect-12,26fd4d58431bb166,9186da3e49e55779] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 3016178, data is empty. 2025-09-02 13:36:56.380 WARN 13484 --- [scanner-12209474-1,26fd4d58431bb166,3e92f605fe8a0daa] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:36:56.392 INFO 13484 --- [alarm-detect-19,26fd4d58431bb166,20e4dce857946288] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 3016178, data is empty. 2025-09-02 13:36:56.392 INFO 13484 --- [alarm-detect-10,26fd4d58431bb166,0c90ed783561d013] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 3016178, data is empty. 2025-09-02 13:36:56.427 INFO 13484 --- [ocp-async-6,a90c064ce6897060,7b946289110c4c70] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:56.459 INFO 13484 --- [alarm-detect-1,26fd4d58431bb166,21cc5b991a2c9214] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 3016178, data is empty. 2025-09-02 13:36:56.539 INFO 13484 --- [ocp-schedules-12,9ae0c3f3e0c06509,f1f4e93899b00255] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-09-02 13:36:56.552 INFO 13484 --- [ocp-schedules-11,f95e5c7ebb727a8c,cb55df124cc675e1] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:36:56.552 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=172.19.62.154, originalStatus=ONLINE 2025-09-02 13:36:56.552 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=172.19.62.156, originalStatus=ONLINE 2025-09-02 13:36:56.553 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=172.17.172.84, originalStatus=ONLINE 2025-09-02 13:36:56.553 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=172.17.172.86, originalStatus=ONLINE 2025-09-02 13:36:56.553 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=172.17.172.85, originalStatus=ONLINE 2025-09-02 13:36:56.557 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:36:56.557 INFO 13484 --- [ocp-schedules-24,fc114e7f10b2fd3d,6e3eeb005b279d73] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:36:56.557 INFO 13484 --- [host-check-agent-status1,fc114e7f10b2fd3d,f15c18747002c0b7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=172.19.62.154, originalStatus=ONLINE 2025-09-02 13:36:56.558 INFO 13484 --- [host-check-agent-status2,fc114e7f10b2fd3d,411052f5e087ab33] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=172.19.62.156, originalStatus=ONLINE 2025-09-02 13:36:56.558 INFO 13484 --- [host-check-agent-status3,fc114e7f10b2fd3d,83916348103e87bb] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=172.17.172.84, originalStatus=ONLINE 2025-09-02 13:36:56.558 INFO 13484 --- [host-check-agent-status4,fc114e7f10b2fd3d,1c38874db2900b3b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=172.17.172.86, originalStatus=ONLINE 2025-09-02 13:36:56.558 INFO 13484 --- [host-check-agent-status5,fc114e7f10b2fd3d,3099352b136da858] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=172.17.172.85, originalStatus=ONLINE 2025-09-02 13:36:56.558 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:36:56.558 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:36:56.558 INFO 13484 --- [ocp-schedules-8,400906252dbe77be,61bfee524b9b27bc] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-09-02 13:36:56.560 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:36:56.560 INFO 13484 --- [ocp-schedules-8,400906252dbe77be,61bfee524b9b27bc] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:56.560 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:36:56.560 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:36:56.562 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:36:56.562 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:36:56.562 INFO 13484 --- [host-check-agent-status2,fc114e7f10b2fd3d,411052f5e087ab33] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:36:56.564 INFO 13484 --- [ocp-schedules-10,fd5dcd44feac7713,ec999168fe9952e3] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:36:56.564 INFO 13484 --- [host-check-agent-status2,fc114e7f10b2fd3d,411052f5e087ab33] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:36:56.564 INFO 13484 --- [host-check-clockdiff1,fd5dcd44feac7713,cb60786133c0a970] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.19.62.154], timeoutMillis=20000 2025-09-02 13:36:56.565 INFO 13484 --- [host-check-agent-status5,fc114e7f10b2fd3d,3099352b136da858] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:36:56.565 INFO 13484 --- [ocp-schedules-14,18ce1a61476631bf,117cf13d0c21df16] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-09-02 13:36:56.565 INFO 13484 --- [host-check-clockdiff2,fd5dcd44feac7713,52768a22d85c0f30] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.19.62.156], timeoutMillis=20000 2025-09-02 13:36:56.565 INFO 13484 --- [host-check-agent-status4,fc114e7f10b2fd3d,1c38874db2900b3b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:36:56.565 INFO 13484 --- [ocp-schedules-1,8e3199c206df8db6,969c6fff73df9fe5] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-09-02 13:36:56.566 INFO 13484 --- [host-check-clockdiff4,fd5dcd44feac7713,f7e2180471acea25] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.86], timeoutMillis=20000 2025-09-02 13:36:56.567 INFO 13484 --- [ocp-schedules-14,18ce1a61476631bf,117cf13d0c21df16] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:56.567 INFO 13484 --- [ocp-schedules-1,8e3199c206df8db6,969c6fff73df9fe5] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:56.568 INFO 13484 --- [host-check-agent-status3,fc114e7f10b2fd3d,83916348103e87bb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:36:56.577 INFO 13484 --- [host-check-clockdiff3,fd5dcd44feac7713,99dde34ced94931b] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.84], timeoutMillis=20000 2025-09-02 13:36:56.577 INFO 13484 --- [ocp-schedules-14,18ce1a61476631bf,117cf13d0c21df16] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=1, successCount=1 2025-09-02 13:36:56.578 INFO 13484 --- [host-check-agent-status3,fc114e7f10b2fd3d,83916348103e87bb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:36:56.578 INFO 13484 --- [host-check-agent-status5,fc114e7f10b2fd3d,3099352b136da858] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:36:56.578 INFO 13484 --- [host-check-agent-status4,fc114e7f10b2fd3d,1c38874db2900b3b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:36:56.578 INFO 13484 --- [host-check-clockdiff5,fd5dcd44feac7713,609fcc89d8219e3c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.85], timeoutMillis=20000 2025-09-02 13:36:56.582 INFO 13484 --- [host-check-agent-status1,fc114e7f10b2fd3d,f15c18747002c0b7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:36:56.582 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:36:56.584 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:36:56.584 INFO 13484 --- [host-check-agent-status1,fc114e7f10b2fd3d,f15c18747002c0b7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:36:56.587 INFO 13484 --- [ocp-schedules-14,909f97425fdb5f05,ce94c42c008857f5] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-09-02 13:36:56.589 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62888/api/v1/time, params:null 2025-09-02 13:36:56.594 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=172.17.172.85, timestamp=2025-09-02T13:36:56.592497879+08:00 2025-09-02 13:36:56.594 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-09-02T13:36:56.594+08:00 2025-09-02 13:36:56.596 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-09-02T13:36:56.594+08:00 and status AVAILABLE 2025-09-02 13:36:56.606 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-09-02 13:36:56.610 INFO 13484 --- [host-check-clockdiff1,fd5dcd44feac7713,cb60786133c0a970] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791416 0 0 , error= 2025-09-02 13:36:56.622 INFO 13484 --- [host-check-clockdiff2,fd5dcd44feac7713,52768a22d85c0f30] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791416 0 0 , error= 2025-09-02 13:36:56.622 INFO 13484 --- [host-check-clockdiff4,fd5dcd44feac7713,f7e2180471acea25] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791416 -1 -1 , error= 2025-09-02 13:36:56.629 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62888/api/v1/time, params:null 2025-09-02 13:36:56.629 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62888/api/v1/time, params:null 2025-09-02 13:36:56.629 INFO 13484 --- [ocp-schedules-14,cbb2419fad7040ff,acda237f5b29aa80] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster root server. 2025-09-02 13:36:56.631 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=172.19.62.156, timestamp=2025-09-02T13:36:56.630591868+08:00 2025-09-02 13:36:56.632 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=172.17.172.84, timestamp=2025-09-02T13:36:56.631153248+08:00 2025-09-02 13:36:56.632 INFO 13484 --- [host-check-agent-status4,fc114e7f10b2fd3d,1c38874db2900b3b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62888/api/v1/agent/status, params:null 2025-09-02 13:36:56.632 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-09-02T13:36:56.631+08:00 2025-09-02 13:36:56.632 INFO 13484 --- [ocp-schedules-14,cbb2419fad7040ff,acda237f5b29aa80] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:36:56.633 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-09-02T13:36:56.632+08:00 2025-09-02 13:36:56.634 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:36:56.636 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-09-02T13:36:56.632+08:00 and status AVAILABLE 2025-09-02 13:36:56.636 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-09-02T13:36:56.631+08:00 and status AVAILABLE 2025-09-02 13:36:56.636 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:36:56.636 INFO 13484 --- [host-check-agent-status4,fc114e7f10b2fd3d,1c38874db2900b3b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.86 2025-09-02 13:36:56.636 INFO 13484 --- [host-check-agent-status4,fc114e7f10b2fd3d,1c38874db2900b3b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.86 2025-09-02 13:36:56.636 INFO 13484 --- [host-check-agent-status4,fc114e7f10b2fd3d,1c38874db2900b3b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.86 2025-09-02 13:36:56.640 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62888/api/v1/time, params:null 2025-09-02 13:36:56.640 INFO 13484 --- [host-check-agent-status5,fc114e7f10b2fd3d,3099352b136da858] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62888/api/v1/agent/status, params:null 2025-09-02 13:36:56.640 INFO 13484 --- [host-check-agent-status3,fc114e7f10b2fd3d,83916348103e87bb] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62888/api/v1/agent/status, params:null 2025-09-02 13:36:56.640 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=172.17.172.86, timestamp=2025-09-02T13:36:56.640249212+08:00 2025-09-02 13:36:56.641 INFO 13484 --- [host-check-agent-status1,fc114e7f10b2fd3d,f15c18747002c0b7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62888/api/v1/agent/status, params:null 2025-09-02 13:36:56.641 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-09-02T13:36:56.641+08:00 2025-09-02 13:36:56.642 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62888/api/v1/time, params:null 2025-09-02 13:36:56.642 INFO 13484 --- [host-check-agent-status2,fc114e7f10b2fd3d,411052f5e087ab33] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62888/api/v1/agent/status, params:null 2025-09-02 13:36:56.643 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-09-02 13:36:56.643 INFO 13484 --- [host-check-agent-status5,fc114e7f10b2fd3d,3099352b136da858] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.85 2025-09-02 13:36:56.643 INFO 13484 --- [host-check-agent-status5,fc114e7f10b2fd3d,3099352b136da858] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.85 2025-09-02 13:36:56.643 INFO 13484 --- [host-check-agent-status5,fc114e7f10b2fd3d,3099352b136da858] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.85 2025-09-02 13:36:56.643 INFO 13484 --- [host-check-agent-status3,fc114e7f10b2fd3d,83916348103e87bb] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.84 2025-09-02 13:36:56.643 INFO 13484 --- [host-check-agent-status3,fc114e7f10b2fd3d,83916348103e87bb] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.84 2025-09-02 13:36:56.643 INFO 13484 --- [host-check-agent-status3,fc114e7f10b2fd3d,83916348103e87bb] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.84 2025-09-02 13:36:56.644 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=172.19.62.154, timestamp=2025-09-02T13:36:56.644299071+08:00 2025-09-02 13:36:56.645 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-09-02T13:36:56.641+08:00 and status AVAILABLE 2025-09-02 13:36:56.645 INFO 13484 --- [host-check-agent-status2,fc114e7f10b2fd3d,411052f5e087ab33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.19.62.156 2025-09-02 13:36:56.646 INFO 13484 --- [host-check-agent-status2,fc114e7f10b2fd3d,411052f5e087ab33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.19.62.156 2025-09-02 13:36:56.646 INFO 13484 --- [host-check-agent-status2,fc114e7f10b2fd3d,411052f5e087ab33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.19.62.156 2025-09-02 13:36:56.646 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-09-02 13:36:56.646 INFO 13484 --- [host-check-agent-status1,fc114e7f10b2fd3d,f15c18747002c0b7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.19.62.154 2025-09-02 13:36:56.646 INFO 13484 --- [host-check-agent-status1,fc114e7f10b2fd3d,f15c18747002c0b7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.19.62.154 2025-09-02 13:36:56.646 INFO 13484 --- [host-check-agent-status1,fc114e7f10b2fd3d,f15c18747002c0b7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.19.62.154 2025-09-02 13:36:56.647 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-09-02T13:36:56.644+08:00 2025-09-02 13:36:56.649 INFO 13484 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.650 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-09-02T13:36:56.644+08:00 and status AVAILABLE 2025-09-02 13:36:56.650 INFO 13484 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-09-02 13:36:56.651 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:36:56.652 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-09-02 13:36:56.653 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:36:56.654 INFO 13484 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.3.5.3 2025-09-02 13:36:56.654 INFO 13484 --- [ocp-schedules-8,400906252dbe77be,61bfee524b9b27bc] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:36:56.657 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:36:56.658 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-09-02 13:36:56.658 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:36:56.659 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.659 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:36:56.660 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-09-02 13:36:56.662 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:36:56.665 INFO 13484 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.665 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:36:56.665 INFO 13484 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [obconfig_url] 2025-09-02 13:36:56.668 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:36:56.670 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:36:56.676 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:36:56.679 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,a17e958eaab63209,acc770c57202f989] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=myoceanbase&ObClusterId=1755130302&version=2, client=172.19.62.162, traceId=a17e958eaab63209, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getRootServiceInfo(String,Long,String,Long,boolean,Integer), args=myoceanbase,1755130302,false,2,] 2025-09-02 13:36:56.679 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,a17e958eaab63209,acc770c57202f989] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=172.19.62.162, traceId=a17e958eaab63209, duration=1 ms] 2025-09-02 13:36:56.681 INFO 13484 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.682 INFO 13484 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [rootservice_list] 2025-09-02 13:36:56.686 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:36:56.687 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:36:56.689 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-09-02 13:36:56.693 INFO 13484 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.696 INFO 13484 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [172.17.172.85, 2882] 2025-09-02 13:36:56.699 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-09-02 13:36:56.700 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-09-02 13:36:56.700 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:56.701 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.702 INFO 13484 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.703 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:36:56.705 INFO 13484 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [172.17.172.86, 2882] 2025-09-02 13:36:56.706 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-09-02 13:36:56.706 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-09-02 13:36:56.711 INFO 13484 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.714 INFO 13484 --- [Check root service 1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [172.17.172.84, 2882] 2025-09-02 13:36:56.716 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-09-02 13:36:56.718 INFO 13484 --- [ocp-schedules-14,cbb2419fad7040ff,acda237f5b29aa80] .o.o.s.t.b.c.AbstractShardingClusterTask : Check root service executed completed, allCount=1, successCount=1 2025-09-02 13:36:56.719 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-09-02 13:36:56.721 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:36:56.736 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:36:56.742 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:36:56.751 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-09-02 13:36:56.752 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:56.754 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.754 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-09-02 13:36:56.758 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.758 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-09-02 13:36:56.759 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-09-02 13:36:56.760 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-09-02 13:36:56.761 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-09-02 13:36:56.764 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=BackupAlarmCheckSchedule, method=run 2025-09-02 13:36:56.764 INFO 13484 --- [ocp-schedules-3,0506c987624a620c,a52001e21d9fd318] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2025-09-02 13:36:56.765 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:36:56.770 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-09-02 13:36:56.771 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-09-02 13:36:56.771 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-09-02 13:36:56.774 INFO 13484 --- [ocp-schedules-3,0506c987624a620c,a52001e21d9fd318] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2025-09-02 13:36:56.779 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:36:56.781 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:56.781 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.025724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:36:56.782 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-09-02 13:36:56.789 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-09-02 13:36:56.791 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.b.i.alarm.BackupAlarmService : Backup alarm check schedule. 2025-09-02 13:36:56.794 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:36:56.797 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:36:56.797 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-09-02 13:36:56.804 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:36:56.804 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-09-02 13:36:56.814 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:36:56.815 INFO 13484 --- [host-check-host-status5,f95e5c7ebb727a8c,0030b30b698eb729] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-09-02 13:36:56.818 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.025724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:56.818 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:36:56.822 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-09-02 13:36:56.824 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-09-02 13:36:56.832 INFO 13484 --- [host-check-host-status4,f95e5c7ebb727a8c,0c488ba04a85e8a2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-09-02 13:36:56.835 INFO 13484 --- [host-check-host-status3,f95e5c7ebb727a8c,f3f6e1814ea0c881] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-09-02 13:36:56.835 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:56.835 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27.368133+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:36:56.845 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.846 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:36:56.848 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.848 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:36:56.855 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27.368133+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:56.855 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:36:56.860 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.868 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:56.868 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.373150+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:36:56.874 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-09-02 13:36:56.875 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2025-09-02 13:36:56.884 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-09-02 13:36:56.886 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.373150+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:36:56.889 INFO 13484 --- [ocp-schedules-26,25506f159c683045,380062fb34c83b50] c.o.o.s.common.scheduled.OcpSchedules : Cleanup ocp connection pools. 2025-09-02 13:36:56.889 INFO 13484 --- [ocp-schedules-26,25506f159c683045,380062fb34c83b50] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] clean up the expired connectors. 2025-09-02 13:36:56.900 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:36:56.901 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:56.902 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.903 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-09-02 13:36:56.903 INFO 13484 --- [pool-overview-container-4,94b65e211ee419ce,f85f90f2b2fb4136] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-09-02 13:36:56.905 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-09-02 13:36:56.910 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster myoceanbase 2025-09-02 13:36:56.918 INFO 13484 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 3, sqlPort: 2881 2025-09-02 13:36:56.919 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-09-02 13:36:56.921 INFO 13484 --- [pool-12-thread-6,94b65e211ee419ce,e15b8729cc10e300] c.o.o.p.t.i.ObTransactionDiagServiceImpl : begin to get cluster trans stat 2025-09-02 13:36:56.922 INFO 13484 --- [pool-12-thread-7,94b65e211ee419ce,9d82b4c5408374ea] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get outline history 2025-09-02 13:36:56.922 INFO 13484 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:36:56.922 INFO 13484 --- [pool-overview-container-7,922ae97331cca523,561d66c2231066ca] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.924 INFO 13484 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:36:56.925 INFO 13484 --- [pool-overview-container-7,922ae97331cca523,561d66c2231066ca] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ SVR_IP, SVR_PORT, ZONE, CPU_CAPACITY, CPU_CAPACITY_MAX, CPU_ASSIGNED, CPU_ASSIGNED_MAX, MEM_CAPACITY, MEM_CAPACITY AS MEM_CAPACITY_MAX, MEM_ASSIGNED, MEM_ASSIGNED AS MEM_ASSIGNED_MAX, LOG_DISK_CAPACITY, LOG_DISK_ASSIGNED, LOG_DISK_IN_USE, DATA_DISK_CAPACITY, DATA_DISK_IN_USE FROM oceanbase.GV$OB_SERVERS 2025-09-02 13:36:56.927 INFO 13484 --- [pool-12-thread-8,94b65e211ee419ce,22ad655745210ed9] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get Cluster Sql Stat 2025-09-02 13:36:56.927 INFO 13484 --- [pool-12-thread-8,94b65e211ee419ce,22ad655745210ed9] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get cluster slow Sql Stat 2025-09-02 13:36:56.927 INFO 13484 --- [ForkJoinPool-10-worker-11,,] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-09-02 13:36:56.928 INFO 13484 --- [pool-12-thread-6,94b65e211ee419ce,e15b8729cc10e300] c.o.o.p.t.i.ObTransactionDiagServiceImpl : begin to attach trans id 2025-09-02 13:36:56.928 INFO 13484 --- [pool-12-thread-6,94b65e211ee419ce,e15b8729cc10e300] c.o.o.p.t.i.ObTransactionDiagServiceImpl : finish fetching trans data 2025-09-02 13:36:56.929 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:56.930 INFO 13484 --- [pool-12-thread-5,94b65e211ee419ce,79fd10750a074dbd] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get cluster session stat 2025-09-02 13:36:56.932 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.932 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:36:56.934 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.935 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:36:56.935 INFO 13484 --- [ForkJoinPool-10-worker-4,,] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-09-02 13:36:56.936 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get tenant session stat of myoceanbase 2025-09-02 13:36:56.936 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.o.p.o.i.OasSessionStatServiceImpl : begin to get session of myoceanbase 2025-09-02 13:36:56.941 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:36:56.944 INFO 13484 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-09-02 13:36:56.948 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.961 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.962 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state = 'ACTIVE' 2025-09-02 13:36:56.966 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.966 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.967 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2025-09-02 13:36:56.968 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2025-09-02 13:36:56.970 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2025-09-02 13:36:56.972 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-09-02 13:36:56.980 INFO 13484 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 3, sqlPort: 2881 got ObProcessInfoResult(pid=4794, runPath=/data1/myoceanbase/oceanbase, runUser=root, execute=/data1/myoceanbase/oceanbase/bin/observer, installPath=/data1/myoceanbase/oceanbase) 2025-09-02 13:36:56.982 INFO 13484 --- [pool-12-thread-8,94b65e211ee419ce,22ad655745210ed9] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get cluster anomaly sql Stat 2025-09-02 13:36:56.984 INFO 13484 --- [host-check-host-status1,f95e5c7ebb727a8c,dd335e7efb4b61e4] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-09-02 13:36:56.986 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=woqu, dbUser=root 2025-09-02 13:36:56.987 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:36:56.986517+08:00, lastProcessTime=2025-09-02T13:36:46.985894+08:00 2025-09-02 13:36:56.988 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.989 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2025-09-02 13:36:56.990 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.991 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2025-09-02 13:36:56.994 INFO 13484 --- [pool-12-thread-8,94b65e211ee419ce,22ad655745210ed9] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching cluster sql stat 2025-09-02 13:36:56.997 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:56.998 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:56.999 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-09-02 13:36:57.000 INFO 13484 --- [refresh-tenant-resource-cache-5,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.005 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2025-09-02 13:36:57.006 INFO 13484 --- [refresh-tenant-resource-cache-5,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(60000000) ocp_query */ time_to_usec(t1.MODIFY_TIME) AS UPDATE_TIME, t1.RESOURCE_POOL_ID, t1.NAME, t1.UNIT_COUNT, t1.UNIT_CONFIG_ID, t1.ZONE_LIST, t1.TENANT_ID, t1.REPLICA_TYPE, t2.NAME AS UNIT_CONFIG_NAME, t2.MAX_CPU, t2.MIN_CPU, t2.MEMORY_SIZE AS MAX_MEMORY, t2.MEMORY_SIZE AS MIN_MEMORY, t2.LOG_DISK_SIZE, t2.MAX_IOPS, t2.MIN_IOPS, t2.IOPS_WEIGHT FROM oceanbase.DBA_OB_RESOURCE_POOLS AS t1 JOIN oceanbase.DBA_OB_UNIT_CONFIGS AS t2 ON t1.UNIT_CONFIG_ID = t2.UNIT_CONFIG_ID AND t1.tenant_id = ?, args: [1002] 2025-09-02 13:36:57.008 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.009 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state = 'ACTIVE' 2025-09-02 13:36:57.013 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.014 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2025-09-02 13:36:57.018 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.o.p.o.i.OasSessionStatServiceImpl : begin to get deadlock of myoceanbase 2025-09-02 13:36:57.020 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=myoceanbase, obClusterId=1755130302 2025-09-02 13:36:57.020 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=myoceanbase, obClusterId=1755130302, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"myoceanbase","ObClusterId":1755130302,"ObRegion":"myoceanbase","ObRegionId":1755130302,"ReadonlyRsList":[],"RsList":[{"address":"172.17.172.85:2882","role":"LEADER","sql_port":2881},{"address":"172.17.172.86:2882","role":"FOLLOWER","sql_port":2881},{"address":"172.17.172.84:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1756791174423000}, createTime=2025-08-29T16:02:30+08:00, creator=admin, obVersion=4.3.5.3, packageOperatingSystem=null, loadType=null, attributesJson={"DataDiskPath":"/data1/myoceanbase/oceanbase/store","DiskPathStyle":"OBD","InstallPath":"/data1/myoceanbase/oceanbase","LogDiskPath":"/data1/myoceanbase/oceanbase/store","OperatingSystemUser":"root","RunPath":"/data1/myoceanbase/oceanbase","SqlPort":2881,"SvrPort":2882}, tenantCount=2) 2025-09-02 13:36:57.020 INFO 13484 --- [ocp-schedules-1,8e3199c206df8db6,969c6fff73df9fe5] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:36:57.024 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-09-02 13:36:57.027 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.029 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2025-09-02 13:36:57.033 INFO 13484 --- [host-check-host-status2,f95e5c7ebb727a8c,e5aa5ef7af794c0f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-09-02 13:36:57.037 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=woqu, dbUser=root 2025-09-02 13:36:57.045 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=root 2025-09-02 13:36:57.048 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:57.048 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.049 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.050 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY where report_time > ? and report_time < ?;, args: [2025-08-26 13:36:57, 2025-09-02 13:36:57] 2025-09-02 13:36:57.050 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2025-09-02 13:36:57.050 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.050 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:36:57.051 INFO 13484 --- [pool-overview-container-6,298913317c9b36d4,2a2b851254a1e96c] c.o.ocp.service.iam.user.UserService : user 100 login with organization 10000000 2025-09-02 13:36:57.052 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.053 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:36:57.054 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.055 INFO 13484 --- [ForkJoinPool-10-worker-11,94b65e211ee419ce,90f5c542e7ce27c3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2025-09-02 13:36:57.057 INFO 13484 --- [refresh-tenant-resource-cache-5,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:57.059 INFO 13484 --- [pool-12-thread-7,94b65e211ee419ce,9d82b4c5408374ea] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching outline history 2025-09-02 13:36:57.062 INFO 13484 --- [refresh-tenant-resource-cache-5,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.067 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.071 INFO 13484 --- [refresh-tenant-resource-cache-5,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(5000000) ocp_query */ t1.UNIT_ID, t1.RESOURCE_POOL_ID, t1.UNIT_CONFIG_ID, t1.ZONE, t1.SVR_IP, t1.SVR_PORT, t1.STATUS, t2.REPLICA_TYPE, t2.TENANT_ID, t3.TENANT_NAME, t2.NAME AS RESOURCE_POOL_NAME, t2.MODIFY_TIME AS RESOURCE_POOL_UPDATE_TIME, t1.MIGRATE_FROM_SVR_IP, t1.MIGRATE_FROM_SVR_PORT, t1.MANUAL_MIGRATE FROM oceanbase.DBA_OB_UNITS AS t1 JOIN oceanbase.DBA_OB_RESOURCE_POOLS AS t2 ON t1.RESOURCE_POOL_ID = t2.RESOURCE_POOL_ID JOIN oceanbase.DBA_OB_TENANTS t3 ON t2.TENANT_ID = t3.TENANT_ID WHERE 1=1 AND t2.tenant_id = ?, args: [1002] 2025-09-02 13:36:57.072 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=root 2025-09-02 13:36:57.073 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.074 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [600] 2025-09-02 13:36:57.075 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY where report_time > ? and report_time < ?;, args: [2025-08-26 13:36:57, 2025-09-02 13:36:57] 2025-09-02 13:36:57.079 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.o.p.o.i.OasSessionStatServiceImpl : finish fetching tenant session data of myoceanbase 2025-09-02 13:36:57.079 INFO 13484 --- [ForkJoinPool-10-worker-4,94b65e211ee419ce,f8cce13b3812b671] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching tenant session stat of myoceanbase 2025-09-02 13:36:57.079 INFO 13484 --- [pool-12-thread-5,94b65e211ee419ce,79fd10750a074dbd] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching session stat 2025-09-02 13:36:57.099 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:57.100 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.101 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2025-09-02 13:36:57.102 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2025-09-02 13:36:57.118 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.b.i.alarm.BackupAlarmService : clusterId=1, minDelaySeconds=600, delayTasks=[] 2025-09-02 13:36:57.145 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:57.147 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.147 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:36:57.149 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.149 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:36:57.161 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.161 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:36:56.986517+08:00 2025-09-02 13:36:57.176 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [60] 2025-09-02 13:36:57.213 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=60, timeoutTasks=[] 2025-09-02 13:36:57.241 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:36:57.243 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.243 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:36:57.245 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.245 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:36:57.258 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:36:57.307 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2025-09-02 13:36:57.340 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2025-09-02 13:36:57.355 INFO 13484 --- [ocp-schedules-22,1651685cbe51d299,efa98188fa9aae9c] c.o.o.b.i.alarm.BackupAlarmService : cluster=myoceanbase does not enable secondary backup 2025-09-02 13:36:57.526 INFO 13484 --- [host-check-clockdiff5,fd5dcd44feac7713,609fcc89d8219e3c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791417 -1 -1 , error= 2025-09-02 13:36:57.538 INFO 13484 --- [print-time-series-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Async compute queue max size = 0, current size = 0. 2025-09-02 13:36:57.542 INFO 13484 --- [ocp-schedules-18,05a847c88a58c3b1,5bb2a6dcbde71b34] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:36:57.542 INFO 13484 --- [ocp-schedules-18,05a847c88a58c3b1,5bb2a6dcbde71b34] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2025-09-02 13:36:57.542 INFO 13484 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2025-09-02 13:36:57.542 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2025-09-02 13:36:57.542 INFO 13484 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2025-09-02 13:36:57.542 INFO 13484 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2025-09-02 13:36:57.546 INFO 13484 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:36:57.546 INFO 13484 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:36:57.547 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:36:57.547 INFO 13484 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:36:57.548 INFO 13484 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:36:57.548 INFO 13484 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:36:57.548 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:36:57.549 INFO 13484 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:36:57.549 INFO 13484 --- [ocp-schedules-18,05a847c88a58c3b1,5bb2a6dcbde71b34] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:36:57.550 INFO 13484 --- [ocp-schedules-18,05a847c88a58c3b1,5bb2a6dcbde71b34] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:36:57.567 INFO 13484 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62889/api/v1/log/alarms, params:null 2025-09-02 13:36:57.572 INFO 13484 --- [ForkJoinPool.commonPool-worker-2,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62889/api/v1/log/alarms, params:null 2025-09-02 13:36:57.574 INFO 13484 --- [ForkJoinPool.commonPool-worker-1,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62889/api/v1/log/alarms, params:null 2025-09-02 13:36:57.575 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62889/api/v1/log/alarms, params:null 2025-09-02 13:36:57.579 INFO 13484 --- [ocp-schedules-18,05a847c88a58c3b1,5bb2a6dcbde71b34] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62889/api/v1/log/alarms, params:null 2025-09-02 13:36:57.584 INFO 13484 --- [ocp-schedules-18,05a847c88a58c3b1,5bb2a6dcbde71b34] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms finished. 2025-09-02 13:36:57.851 INFO 13484 --- [ocp-schedules-14,3cf0e7797dce9b81,8100f6a2bccab666] c.o.o.a.service.OcpAlarmSearchService : alarmTargetStat [] 2025-09-02 13:36:57.851 INFO 13484 --- [ocp-schedules-14,3cf0e7797dce9b81,8100f6a2bccab666] c.o.o.a.service.OcpAlarmSearchService : alarmTargetStat [AlarmTargetStat(alarmType=inc_backup_delay, target=alarm_template_id=0:ob_cluster=myoceanbase-1755130302:tenant_name=woqu, statCount=1)] 2025-09-02 13:36:57.852 INFO 13484 --- [ocp-schedules-14,3cf0e7797dce9b81,8100f6a2bccab666] c.o.o.a.service.OcpAlarmSearchService : myoceanbase-1755130302 2025-09-02 13:36:58.252 INFO 13484 --- [ocp-schedules-1,04eb15f6c5cb864c,310757b0454f7e03] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:36:58.323 INFO 13484 --- [prometheus-minute-schedule-1,4fb28b7d38d657a8,58d0cd7c57a89fad] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791420, interval=60 2025-09-02 13:36:58.327 INFO 13484 --- [prometheus-minute-schedule-1,4fb28b7d38d657a8,58d0cd7c57a89fad] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:36:58.365 INFO 13484 --- [host-check-clockdiff3,fd5dcd44feac7713,99dde34ced94931b] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791418 0 0 , error= 2025-09-02 13:36:58.427 INFO 13484 --- [ocp-async-7,5233aa2eeab06263,a269ec18589720fd] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:36:58.959 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:36:58.960 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:36:59.094 INFO 13484 --- [ocp-async-10,74e64071b91a6a43,edaf4b1bcc68339d] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:37:00.428 INFO 13484 --- [ocp-async-1,1f86c459e2f05e06,2e971e3f7132db5e] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:00.790 INFO 13484 --- [ocp-schedules-3,b1c13a3c03abdbea,eaa55beba6f2cbd4] c.o.o.s.c.scheduled.TenantSchedules : Collect all tenant major compaction 2025-09-02 13:37:00.790 INFO 13484 --- [ocp-schedules-3,b1c13a3c03abdbea,eaa55beba6f2cbd4] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2025-09-02 13:37:00.808 INFO 13484 --- [Collect all tenants major compactions1,b1c13a3c03abdbea,49e1e0980b3fd12f] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:myoceanbase, tenant:woqu 2025-09-02 13:37:00.809 INFO 13484 --- [Collect all tenants major compactions2,b1c13a3c03abdbea,3a1b8008cf20e157] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:myoceanbase, tenant:sys 2025-09-02 13:37:00.827 INFO 13484 --- [Collect all tenants major compactions2,b1c13a3c03abdbea,3a1b8008cf20e157] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:00.827 INFO 13484 --- [Collect all tenants major compactions1,b1c13a3c03abdbea,49e1e0980b3fd12f] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:00.828 INFO 13484 --- [Collect all tenants major compactions2,b1c13a3c03abdbea,3a1b8008cf20e157] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-09-02 13:37:00.829 INFO 13484 --- [Collect all tenants major compactions1,b1c13a3c03abdbea,49e1e0980b3fd12f] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2025-09-02 13:37:00.833 INFO 13484 --- [Collect all tenants major compactions1,b1c13a3c03abdbea,49e1e0980b3fd12f] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:myoceanbase, tenant:woqu 2025-09-02 13:37:00.833 INFO 13484 --- [Collect all tenants major compactions2,b1c13a3c03abdbea,3a1b8008cf20e157] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:myoceanbase, tenant:sys 2025-09-02 13:37:00.834 INFO 13484 --- [ocp-schedules-3,b1c13a3c03abdbea,eaa55beba6f2cbd4] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2025-09-02 13:37:00.887 INFO 13484 --- [ocp-schedules-1,e0696defc6d105df,b3f5ba4e3ad13f78] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog instances, size=0 2025-09-02 13:37:00.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,06b06aa055452a60,c4a0ff76a350e524] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=06b06aa055452a60, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:37:00.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,072a402ef948ded8,67f111500e877628] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=072a402ef948ded8, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:37:00.936 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,426579e861873dea,4fecf98d749e6f61] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=426579e861873dea, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:37:00.936 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,06b06aa055452a60,c4a0ff76a350e524] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=06b06aa055452a60, duration=4 ms] 2025-09-02 13:37:00.936 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,072a402ef948ded8,67f111500e877628] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=072a402ef948ded8, duration=4 ms] 2025-09-02 13:37:00.938 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,426579e861873dea,4fecf98d749e6f61] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=426579e861873dea, duration=6 ms] 2025-09-02 13:37:00.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,5ef79c25726baf5d,f2aa2284094f9474] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=5ef79c25726baf5d, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:37:00.938 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,5ef79c25726baf5d,f2aa2284094f9474] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=5ef79c25726baf5d, duration=7 ms] 2025-09-02 13:37:01.022 INFO 13484 --- [ocp-schedules-30,6e1e36fefa520ff1,9e5740676719d12c] c.o.o.s.c.scheduled.ClusterSchedules : Collect all cluster compaction info. 2025-09-02 13:37:01.024 INFO 13484 --- [ocp-schedules-30,6e1e36fefa520ff1,9e5740676719d12c] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:37:01.024 INFO 13484 --- [Collect compaction for all clusters1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-09-02 13:37:01.024 INFO 13484 --- [ocp-schedules-30,6e1e36fefa520ff1,9e5740676719d12c] .o.o.s.t.b.c.AbstractShardingClusterTask : Collect compaction for all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:37:01.323 INFO 13484 --- [prometheus-minute-schedule-1,309b69f20a311f45,090d3acf4e8be075] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:01.328 INFO 13484 --- [prometheus-minute-schedule-1,309b69f20a311f45,090d3acf4e8be075] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:01.960 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:01.960 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:02.064 INFO 13484 --- [ocp-schedules-5,f34cb44c6897d9da,358ea809aec21176] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info 2025-09-02 13:37:02.067 INFO 13484 --- [ocp-schedules-5,f34cb44c6897d9da,358ea809aec21176] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Invalidate clusters [] cache info success. 2025-09-02 13:37:02.067 INFO 13484 --- [ocp-schedules-5,f34cb44c6897d9da,358ea809aec21176] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info success, cached size: 1 2025-09-02 13:37:02.428 INFO 13484 --- [ocp-async-8,c8d72e4607f5951b,08f6171ff505e630] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:02.741 INFO 13484 --- [ocp-schedules-23,2f6274aa12a8f64d,cc79a4a9d624d9f7] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:37:02.743 INFO 13484 --- [ocp-schedules-23,2f6274aa12a8f64d,cc79a4a9d624d9f7] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:37:03.718 INFO 13484 --- [ocp-schedules-16,24bfced1df452e34,5c85b7764de22c10] c.o.o.s.c.scheduled.TenantSchedules : Check all tenant service name conflict 2025-09-02 13:37:03.718 INFO 13484 --- [ocp-schedules-16,24bfced1df452e34,5c85b7764de22c10] .t.CheckAllTenantServiceNameConflictTask : Check all tenant service name conflict. 2025-09-02 13:37:03.761 INFO 13484 --- [ocp-schedules-19,52114f2aa2efa89c,5cd36f15ab4b8518] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-09-02 13:37:03.769 INFO 13484 --- [ocp-schedules-19,52114f2aa2efa89c,5cd36f15ab4b8518] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:37:03.769 INFO 13484 --- [ocp-schedules-19,52114f2aa2efa89c,5cd36f15ab4b8518] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:37:03.997 INFO 13484 --- [ocp-schedules-4,ba8df1d4f1a15ea9,eb1f088bac678f95] c.o.o.s.c.scheduled.TenantSchedules : Check suspend tenant compaction status 2025-09-02 13:37:03.997 INFO 13484 --- [ocp-schedules-4,ba8df1d4f1a15ea9,eb1f088bac678f95] s.t.b.t.CheckSuspendTenantCompactionTask : Check tenant compaction status. 2025-09-02 13:37:04.034 INFO 13484 --- [ocp-schedules-4,ba8df1d4f1a15ea9,eb1f088bac678f95] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:04.035 INFO 13484 --- [ocp-schedules-4,ba8df1d4f1a15ea9,eb1f088bac678f95] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-09-02 13:37:04.323 INFO 13484 --- [prometheus-minute-schedule-1,398de9ad64e0103e,c51b4399307aa02e] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:04.327 INFO 13484 --- [prometheus-minute-schedule-1,398de9ad64e0103e,c51b4399307aa02e] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:04.428 INFO 13484 --- [ocp-async-9,e3042bd2e4fa2816,a503d6ca7669d78a] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:04.924 INFO 13484 --- [ocp-schedules-8,e2f8d2c1d821576c,b7b5caee2e601383] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-09-02 13:37:04.961 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:04.961 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:05.520 INFO 13484 --- [ocp-schedules-18,4cf011cac5a75c0c,a44d410103baefb6] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2025-09-02 13:37:05.520 INFO 13484 --- [ocp-schedules-18,4cf011cac5a75c0c,a44d410103baefb6] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=819357824821161984 2025-09-02 13:37:05.520 INFO 13484 --- [ocp-schedules-18,4cf011cac5a75c0c,a44d410103baefb6] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2025-09-02 13:37:05.520 INFO 13484 --- [ocp-schedules-18,4cf011cac5a75c0c,a44d410103baefb6] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=818961159966617600 2025-09-02 13:37:05.521 INFO 13484 --- [ocp-schedules-18,4cf011cac5a75c0c,a44d410103baefb6] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2025-09-02 13:37:05.521 INFO 13484 --- [ocp-schedules-18,4cf011cac5a75c0c,a44d410103baefb6] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=818784736907816960 2025-09-02 13:37:06.428 INFO 13484 --- [ocp-async-3,3eff7001c9c3d6ca,b0e10fdaf6474893] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:06.469 INFO 13484 --- [ocp-schedules-28,bd6a710bc12cfa9d,137249091f898607] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:37:06.492 WARN 13484 --- [scanner-12209679-1,bd6a710bc12cfa9d,68e0a81645c96156] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:37:06.987 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:37:06.986314+08:00, lastProcessTime=2025-09-02T13:36:56.986517+08:00 2025-09-02 13:37:07.136 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:37:06.986314+08:00 2025-09-02 13:37:07.323 INFO 13484 --- [prometheus-minute-schedule-1,9a1212daece4faea,8d61c85e58bde6ba] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:07.327 INFO 13484 --- [prometheus-minute-schedule-1,9a1212daece4faea,8d61c85e58bde6ba] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:07.961 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:07.962 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:08.256 INFO 13484 --- [ocp-schedules-22,6d7268eea17665fa,633923036682cc67] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:37:08.428 INFO 13484 --- [ocp-async-4,773dafcbe79a670a,b96822215ccbcebf] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:09.094 INFO 13484 --- [ocp-async-6,9bc298f4d3bb3ea3,66f2151e7ee00f5e] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:37:10.004 INFO 13484 --- [ocp-schedules-6,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=2 2025-09-02 13:37:10.004 INFO 13484 --- [ocp-schedules-16,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=2 2025-09-02 13:37:10.020 INFO 13484 --- [Check obproxy1,e8e61b4d50bf4204,f9abb26aed52ebe1] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:37:10.020 INFO 13484 --- [Check obproxy1,e8e61b4d50bf4204,f9abb26aed52ebe1] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:37:10.020 INFO 13484 --- [Check obproxy2,8b633aaf7cf7d0a5,88f58e3317f27011] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:37:10.020 INFO 13484 --- [Check obproxy2,8b633aaf7cf7d0a5,88f58e3317f27011] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:37:10.021 INFO 13484 --- [Collect obproxy2,870d9561b76f88d6,e97c267fd7bfa8d7] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:37:10.021 INFO 13484 --- [Collect obproxy2,870d9561b76f88d6,e97c267fd7bfa8d7] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:37:10.023 INFO 13484 --- [Collect obproxy1,a5125518d7a24595,cf5eb10b952335b2] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:37:10.023 INFO 13484 --- [Collect obproxy1,a5125518d7a24595,cf5eb10b952335b2] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:37:10.044 INFO 13484 --- [Check obproxy2,8b633aaf7cf7d0a5,88f58e3317f27011] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:37:10.045 INFO 13484 --- [Check obproxy1,e8e61b4d50bf4204,f9abb26aed52ebe1] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:37:10.048 INFO 13484 --- [Check obproxy1,e8e61b4d50bf4204,f9abb26aed52ebe1] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-09-02 13:37:10.048 INFO 13484 --- [Check obproxy1,e8e61b4d50bf4204,f9abb26aed52ebe1] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 6, need to sync to meta: false 2025-09-02 13:37:10.050 INFO 13484 --- [Check obproxy2,8b633aaf7cf7d0a5,88f58e3317f27011] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-09-02 13:37:10.050 INFO 13484 --- [Check obproxy2,8b633aaf7cf7d0a5,88f58e3317f27011] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 5, need to sync to meta: false 2025-09-02 13:37:10.057 INFO 13484 --- [Collect obproxy1,a5125518d7a24595,cf5eb10b952335b2] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:37:10.059 INFO 13484 --- [Collect obproxy2,870d9561b76f88d6,e97c267fd7bfa8d7] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:37:10.061 INFO 13484 --- [Collect obproxy1,a5125518d7a24595,cf5eb10b952335b2] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:37:10.062 INFO 13484 --- [Collect obproxy2,870d9561b76f88d6,e97c267fd7bfa8d7] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:37:10.063 INFO 13484 --- [Collect obproxy1,a5125518d7a24595,cf5eb10b952335b2] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:37:10.065 INFO 13484 --- [Collect obproxy2,870d9561b76f88d6,e97c267fd7bfa8d7] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:37:10.151 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,82bf40aeeac652f3,bdabc928b976c295] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=82bf40aeeac652f3, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:37:10.151 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,08e2680fbadd42a0,5c0840e7e915267e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=08e2680fbadd42a0, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:37:10.151 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,3a610d5f165b77b3,2bde9eeda78d7ab7] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=3a610d5f165b77b3, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:37:10.152 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,82bf40aeeac652f3,bdabc928b976c295] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=82bf40aeeac652f3, duration=5 ms] 2025-09-02 13:37:10.152 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,3a610d5f165b77b3,2bde9eeda78d7ab7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=3a610d5f165b77b3, duration=4 ms] 2025-09-02 13:37:10.152 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,08e2680fbadd42a0,5c0840e7e915267e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=08e2680fbadd42a0, duration=5 ms] 2025-09-02 13:37:10.159 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,bd42fb7b1f0f36a2,c17bffeb4eed2285] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=bd42fb7b1f0f36a2, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:37:10.162 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,bd42fb7b1f0f36a2,c17bffeb4eed2285] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=bd42fb7b1f0f36a2, duration=14 ms] 2025-09-02 13:37:10.323 INFO 13484 --- [prometheus-minute-schedule-1,a7cc747183dd33f2,57c755f6f6d349f7] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:10.327 INFO 13484 --- [prometheus-minute-schedule-1,a7cc747183dd33f2,57c755f6f6d349f7] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:10.428 INFO 13484 --- [ocp-async-9,1877faeb6dbc7944,7670af5da98d9dfe] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:10.551 INFO 13484 --- [async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 11, ruleSize = 11. 2025-09-02 13:37:10.962 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:10.963 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:12.429 INFO 13484 --- [ocp-async-2,e57836fa46f94de1,06d27a533a4bb4df] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:12.750 INFO 13484 --- [ocp-schedules-15,2d152bcbaff79a87,c5b184f740a5427a] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:37:12.752 INFO 13484 --- [ocp-schedules-15,2d152bcbaff79a87,c5b184f740a5427a] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:37:13.323 INFO 13484 --- [prometheus-minute-schedule-1,7921accb4f7e1ec4,1899ead0afbe4f17] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:13.329 INFO 13484 --- [prometheus-minute-schedule-1,7921accb4f7e1ec4,1899ead0afbe4f17] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:13.519 INFO 13484 --- [metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1756791360, nodeCount=1000, affectRows=1000 2025-09-02 13:37:13.520 INFO 13484 --- [metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1756791360, nodeCount=1000, affectRows=1000 2025-09-02 13:37:13.524 INFO 13484 --- [metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1756791360, nodeCount=996, validCount=994 2025-09-02 13:37:13.524 INFO 13484 --- [metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1756791360, nodeCount=998, validCount=996 2025-09-02 13:37:13.529 INFO 13484 --- [metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1756791360, nodeCount=678, affectRows=678 2025-09-02 13:37:13.532 INFO 13484 --- [metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1756791360, nodeCount=676, validCount=674 2025-09-02 13:37:13.674 INFO 13484 --- [ocp-schedules-6,5d499c993b051132,1ba8979fc69c42f7] c.o.o.alarm.service.OcpAlarmServiceImpl : fix alarm target, alarms size 8 2025-09-02 13:37:13.675 INFO 13484 --- [ocp-schedules-21,63be28e69a9f7710,4d77f0a9ee60194a] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration auto replace status 2025-09-02 13:37:13.677 INFO 13484 --- [ocp-schedules-28,4c0425435412062e,24fbf98ee3384b06] c.o.o.s.c.scheduled.ClusterSchedules : Check certificate expired. 2025-09-02 13:37:13.963 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:13.964 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:14.429 INFO 13484 --- [ocp-async-4,40678a7acc2fa61a,3d648deade7ffa50] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:15.776 INFO 13484 --- [metric-archive-async-6,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1756791360, nodeCount=385, affectRows=385 2025-09-02 13:37:16.323 INFO 13484 --- [prometheus-minute-schedule-1,1a43196d6fdc3901,c30c620f3d1e20d2] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:16.328 INFO 13484 --- [prometheus-minute-schedule-1,1a43196d6fdc3901,c30c620f3d1e20d2] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:16.429 INFO 13484 --- [ocp-async-7,9625954bc24d137a,e06f8639a602a69c] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:16.554 INFO 13484 --- [ocp-schedules-9,d1495dfe962bf950,6e43e0c629d46a8d] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:37:16.588 WARN 13484 --- [scanner-12210451-1,d1495dfe962bf950,4f683681fc2eafe4] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:37:16.590 INFO 13484 --- [alarm-detect-13,d1495dfe962bf950,26fb8164c6e45484] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 3016178, data is empty. 2025-09-02 13:37:16.618 INFO 13484 --- [alarm-detect-15,d1495dfe962bf950,340d9f898347b25a] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 3016178, data is empty. 2025-09-02 13:37:16.618 INFO 13484 --- [alarm-detect-8,d1495dfe962bf950,6862973c20a5a450] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 3016178, data is empty. 2025-09-02 13:37:16.670 INFO 13484 --- [alarm-detect-15,d1495dfe962bf950,75361cba26d8a1c4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 3016178, data is empty. 2025-09-02 13:37:16.964 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:16.965 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:16.985 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:37:16.984849+08:00, lastProcessTime=2025-09-02T13:37:06.986314+08:00 2025-09-02 13:37:17.151 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:37:16.984849+08:00 2025-09-02 13:37:18.104 INFO 13484 --- [ocp-schedules-22,d447a0fc75c661a1,18664e569c2f418d] c.o.o.s.common.scheduled.PerfSchedules : Oas sql curing, clusters=[] 2025-09-02 13:37:18.261 INFO 13484 --- [ocp-schedules-30,434e195bdad0a253,9ee5b032e5a3e64b] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:37:18.429 INFO 13484 --- [ocp-async-3,9169cd0764bf2977,d519270e5116f953] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:19.094 INFO 13484 --- [ocp-async-5,881b9b91ee07db5d,49cafe32ff325294] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:37:19.323 INFO 13484 --- [prometheus-minute-schedule-1,0e866c32cc069b9d,d72a0e3774dc024f] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:19.327 INFO 13484 --- [prometheus-minute-schedule-1,0e866c32cc069b9d,d72a0e3774dc024f] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:19.966 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:19.967 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:20.429 INFO 13484 --- [ocp-async-10,0fd1692d9bc047bf,08e251a421247da8] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:20.685 INFO 13484 --- [ocp-schedules-18,474171f0f5bd42ff,58ce8da0a480b453] c.o.o.s.common.scheduled.PerfSchedules : Collect trans stat for alarm, time=2025-09-02T13:37:20.685+08:00, clusters=[1] 2025-09-02 13:37:20.687 INFO 13484 --- [ocp-schedules-18,474171f0f5bd42ff,58ce8da0a480b453] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster myoceanbase, trans stat result is empty. 2025-09-02 13:37:20.927 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,6cf093da4831b3c7,f27861a37d7fe77c] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=6cf093da4831b3c7, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:37:20.927 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,8f9918bfe38f4285,e25be780bc84fd33] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=8f9918bfe38f4285, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:37:20.927 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,dc579847bd35a384,f36a764ffc6cba32] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=dc579847bd35a384, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:37:20.927 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,c9fb33c9917c0b50,9a1f5ea05b30d442] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=c9fb33c9917c0b50, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:37:20.928 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,6cf093da4831b3c7,f27861a37d7fe77c] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=6cf093da4831b3c7, duration=4 ms] 2025-09-02 13:37:20.928 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,c9fb33c9917c0b50,9a1f5ea05b30d442] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=c9fb33c9917c0b50, duration=4 ms] 2025-09-02 13:37:20.929 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,dc579847bd35a384,f36a764ffc6cba32] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=dc579847bd35a384, duration=6 ms] 2025-09-02 13:37:20.936 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,8f9918bfe38f4285,e25be780bc84fd33] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=8f9918bfe38f4285, duration=13 ms] 2025-09-02 13:37:22.323 INFO 13484 --- [prometheus-minute-schedule-1,196580d009ba455e,3bfd061135fec2bb] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:22.328 INFO 13484 --- [prometheus-minute-schedule-1,196580d009ba455e,3bfd061135fec2bb] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:22.430 INFO 13484 --- [ocp-async-7,9db6bcd76c1b0cf3,98d8f426f30e1f19] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:22.760 INFO 13484 --- [ocp-schedules-15,ce94e371022be106,0c9cbf94010fb730] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:37:22.763 INFO 13484 --- [ocp-schedules-15,ce94e371022be106,0c9cbf94010fb730] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:37:22.967 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:22.968 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:23.772 INFO 13484 --- [ocp-schedules-16,735b544dc3a23aa5,dbf7c8bf6886caa2] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-09-02 13:37:23.779 INFO 13484 --- [ocp-schedules-16,735b544dc3a23aa5,dbf7c8bf6886caa2] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:37:23.779 INFO 13484 --- [ocp-schedules-16,735b544dc3a23aa5,dbf7c8bf6886caa2] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:37:24.430 INFO 13484 --- [ocp-async-3,48420b425e13a78d,1a1ede66c8a63946] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:25.323 INFO 13484 --- [prometheus-minute-schedule-1,65d796ac8a659812,2c82253e1eb0c2ba] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:25.328 INFO 13484 --- [prometheus-minute-schedule-1,65d796ac8a659812,2c82253e1eb0c2ba] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:25.968 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:25.969 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:26.431 INFO 13484 --- [ocp-async-1,1827006711e3716f,91cd140d68f5cc4b] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:26.539 INFO 13484 --- [ocp-schedules-11,0292b1c5c79dad8a,42561bfee2ced636] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-09-02 13:37:26.552 INFO 13484 --- [ocp-schedules-13,3dacd66de410e247,c8a00b5ae74ac346] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:37:26.553 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=172.19.62.154, originalStatus=ONLINE 2025-09-02 13:37:26.553 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=172.19.62.156, originalStatus=ONLINE 2025-09-02 13:37:26.554 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=172.17.172.84, originalStatus=ONLINE 2025-09-02 13:37:26.554 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=172.17.172.86, originalStatus=ONLINE 2025-09-02 13:37:26.554 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=172.17.172.85, originalStatus=ONLINE 2025-09-02 13:37:26.557 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:37:26.557 INFO 13484 --- [ocp-schedules-19,c1401019e5e22d72,1e7000bca194bfd3] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:37:26.557 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:37:26.557 INFO 13484 --- [host-check-agent-status1,c1401019e5e22d72,87f1720017934666] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=172.19.62.154, originalStatus=ONLINE 2025-09-02 13:37:26.558 INFO 13484 --- [host-check-agent-status2,c1401019e5e22d72,c4c3a51f714fd221] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=172.19.62.156, originalStatus=ONLINE 2025-09-02 13:37:26.558 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:37:26.558 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:37:26.558 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:37:26.558 INFO 13484 --- [host-check-agent-status3,c1401019e5e22d72,d61594dc6d07307f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=172.17.172.84, originalStatus=ONLINE 2025-09-02 13:37:26.558 INFO 13484 --- [host-check-agent-status4,c1401019e5e22d72,1cadc61983f5c741] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=172.17.172.86, originalStatus=ONLINE 2025-09-02 13:37:26.558 INFO 13484 --- [host-check-agent-status5,c1401019e5e22d72,235cc76cd3d4a7cd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=172.17.172.85, originalStatus=ONLINE 2025-09-02 13:37:26.558 INFO 13484 --- [ocp-schedules-17,7d6bf2237570a558,06f311d8a606986c] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-09-02 13:37:26.559 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:37:26.559 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:37:26.560 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:37:26.561 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:37:26.561 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:37:26.561 INFO 13484 --- [ocp-schedules-17,7d6bf2237570a558,06f311d8a606986c] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:37:26.565 INFO 13484 --- [host-check-agent-status1,c1401019e5e22d72,87f1720017934666] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:37:26.565 INFO 13484 --- [host-check-agent-status4,c1401019e5e22d72,1cadc61983f5c741] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:37:26.566 INFO 13484 --- [host-check-agent-status2,c1401019e5e22d72,c4c3a51f714fd221] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:37:26.567 INFO 13484 --- [host-check-agent-status4,c1401019e5e22d72,1cadc61983f5c741] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:37:26.567 INFO 13484 --- [host-check-agent-status1,c1401019e5e22d72,87f1720017934666] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:37:26.567 INFO 13484 --- [host-check-agent-status2,c1401019e5e22d72,c4c3a51f714fd221] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:37:26.567 INFO 13484 --- [host-check-agent-status3,c1401019e5e22d72,d61594dc6d07307f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:37:26.568 INFO 13484 --- [host-check-agent-status5,c1401019e5e22d72,235cc76cd3d4a7cd] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:37:26.569 INFO 13484 --- [host-check-agent-status3,c1401019e5e22d72,d61594dc6d07307f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:37:26.569 INFO 13484 --- [ocp-schedules-26,f9b81b32964225b5,db381a259d6695df] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-09-02 13:37:26.569 INFO 13484 --- [host-check-agent-status5,c1401019e5e22d72,235cc76cd3d4a7cd] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:37:26.570 INFO 13484 --- [ocp-schedules-26,f9b81b32964225b5,db381a259d6695df] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:37:26.571 INFO 13484 --- [ocp-schedules-20,0ad4b2692f1472f8,35a62fb2c2616b83] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-09-02 13:37:26.571 INFO 13484 --- [ocp-schedules-26,f9b81b32964225b5,db381a259d6695df] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=1, successCount=1 2025-09-02 13:37:26.573 INFO 13484 --- [ocp-schedules-16,bd450de42c26563c,13309dd9079e449c] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:37:26.573 INFO 13484 --- [ocp-schedules-20,0ad4b2692f1472f8,35a62fb2c2616b83] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:37:26.575 INFO 13484 --- [host-check-clockdiff1,bd450de42c26563c,940231cae4f9ae50] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.19.62.154], timeoutMillis=20000 2025-09-02 13:37:26.577 INFO 13484 --- [host-check-clockdiff2,bd450de42c26563c,32d56e425573c9d2] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.19.62.156], timeoutMillis=20000 2025-09-02 13:37:26.579 INFO 13484 --- [host-check-clockdiff4,bd450de42c26563c,f1f1c31722636264] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.86], timeoutMillis=20000 2025-09-02 13:37:26.579 INFO 13484 --- [host-check-clockdiff3,bd450de42c26563c,8b4876b888cac3b7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.84], timeoutMillis=20000 2025-09-02 13:37:26.579 INFO 13484 --- [host-check-clockdiff5,bd450de42c26563c,23a4ac6d8aeb5a35] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.85], timeoutMillis=20000 2025-09-02 13:37:26.585 INFO 13484 --- [ocp-schedules-22,ca230c9db1f27c0f,454132c11e28eeae] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-09-02 13:37:26.601 INFO 13484 --- [host-check-clockdiff2,bd450de42c26563c,32d56e425573c9d2] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791446 -1 -1 , error= 2025-09-02 13:37:26.602 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62888/api/v1/time, params:null 2025-09-02 13:37:26.604 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=172.19.62.154, timestamp=2025-09-02T13:37:26.603799115+08:00 2025-09-02 13:37:26.604 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62888/api/v1/time, params:null 2025-09-02 13:37:26.604 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62888/api/v1/time, params:null 2025-09-02 13:37:26.618 INFO 13484 --- [host-check-clockdiff5,bd450de42c26563c,23a4ac6d8aeb5a35] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791446 -1 -1 , error= 2025-09-02 13:37:26.618 INFO 13484 --- [host-check-clockdiff3,bd450de42c26563c,8b4876b888cac3b7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791446 0 0 , error= 2025-09-02 13:37:26.619 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-09-02T13:37:26.604+08:00 2025-09-02 13:37:26.619 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=172.17.172.85, timestamp=2025-09-02T13:37:26.618689003+08:00 2025-09-02 13:37:26.620 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=172.19.62.156, timestamp=2025-09-02T13:37:26.618947905+08:00 2025-09-02 13:37:26.627 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-09-02T13:37:26.619+08:00 2025-09-02 13:37:26.628 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-09-02T13:37:26.604+08:00 and status AVAILABLE 2025-09-02 13:37:26.629 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-09-02T13:37:26.619+08:00 and status AVAILABLE 2025-09-02 13:37:26.625 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-09-02T13:37:26.620+08:00 2025-09-02 13:37:26.631 INFO 13484 --- [host-check-clockdiff4,bd450de42c26563c,f1f1c31722636264] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791446 0 0 , error= 2025-09-02 13:37:26.632 INFO 13484 --- [host-check-agent-status2,c1401019e5e22d72,c4c3a51f714fd221] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62888/api/v1/agent/status, params:null 2025-09-02 13:37:26.633 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-09-02 13:37:26.635 INFO 13484 --- [host-check-agent-status2,c1401019e5e22d72,c4c3a51f714fd221] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.19.62.156 2025-09-02 13:37:26.635 INFO 13484 --- [host-check-agent-status2,c1401019e5e22d72,c4c3a51f714fd221] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.19.62.156 2025-09-02 13:37:26.635 INFO 13484 --- [host-check-agent-status2,c1401019e5e22d72,c4c3a51f714fd221] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.19.62.156 2025-09-02 13:37:26.638 INFO 13484 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:26.638 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62888/api/v1/time, params:null 2025-09-02 13:37:26.639 INFO 13484 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-09-02 13:37:26.639 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62888/api/v1/time, params:null 2025-09-02 13:37:26.641 INFO 13484 --- [host-check-agent-status3,c1401019e5e22d72,d61594dc6d07307f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62888/api/v1/agent/status, params:null 2025-09-02 13:37:26.641 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=172.17.172.86, timestamp=2025-09-02T13:37:26.641250391+08:00 2025-09-02 13:37:26.642 INFO 13484 --- [host-check-agent-status1,c1401019e5e22d72,87f1720017934666] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62888/api/v1/agent/status, params:null 2025-09-02 13:37:26.643 INFO 13484 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.3.5.3 2025-09-02 13:37:26.643 INFO 13484 --- [ocp-schedules-17,7d6bf2237570a558,06f311d8a606986c] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:37:26.644 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=172.17.172.84, timestamp=2025-09-02T13:37:26.643456002+08:00 2025-09-02 13:37:26.644 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-09-02T13:37:26.644+08:00 2025-09-02 13:37:26.645 INFO 13484 --- [host-check-agent-status3,c1401019e5e22d72,d61594dc6d07307f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.84 2025-09-02 13:37:26.645 INFO 13484 --- [host-check-agent-status3,c1401019e5e22d72,d61594dc6d07307f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.84 2025-09-02 13:37:26.645 INFO 13484 --- [host-check-agent-status3,c1401019e5e22d72,d61594dc6d07307f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.84 2025-09-02 13:37:26.645 INFO 13484 --- [host-check-agent-status1,c1401019e5e22d72,87f1720017934666] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.19.62.154 2025-09-02 13:37:26.645 INFO 13484 --- [host-check-agent-status1,c1401019e5e22d72,87f1720017934666] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.19.62.154 2025-09-02 13:37:26.645 INFO 13484 --- [host-check-agent-status1,c1401019e5e22d72,87f1720017934666] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.19.62.154 2025-09-02 13:37:26.646 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:37:26.647 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-09-02T13:37:26.644+08:00 and status AVAILABLE 2025-09-02 13:37:26.647 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-09-02T13:37:26.642+08:00 2025-09-02 13:37:26.647 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-09-02T13:37:26.620+08:00 and status AVAILABLE 2025-09-02 13:37:26.648 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-09-02 13:37:26.649 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:37:26.650 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-09-02T13:37:26.642+08:00 and status AVAILABLE 2025-09-02 13:37:26.651 INFO 13484 --- [host-check-agent-status4,c1401019e5e22d72,1cadc61983f5c741] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62888/api/v1/agent/status, params:null 2025-09-02 13:37:26.651 INFO 13484 --- [host-check-agent-status5,c1401019e5e22d72,235cc76cd3d4a7cd] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62888/api/v1/agent/status, params:null 2025-09-02 13:37:26.654 INFO 13484 --- [host-check-agent-status4,c1401019e5e22d72,1cadc61983f5c741] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.86 2025-09-02 13:37:26.654 INFO 13484 --- [host-check-agent-status4,c1401019e5e22d72,1cadc61983f5c741] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.86 2025-09-02 13:37:26.654 INFO 13484 --- [host-check-agent-status4,c1401019e5e22d72,1cadc61983f5c741] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.86 2025-09-02 13:37:26.654 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-09-02 13:37:26.654 INFO 13484 --- [host-check-agent-status5,c1401019e5e22d72,235cc76cd3d4a7cd] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.85 2025-09-02 13:37:26.654 INFO 13484 --- [host-check-agent-status5,c1401019e5e22d72,235cc76cd3d4a7cd] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.85 2025-09-02 13:37:26.654 INFO 13484 --- [host-check-agent-status5,c1401019e5e22d72,235cc76cd3d4a7cd] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.85 2025-09-02 13:37:26.656 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:37:26.658 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-09-02 13:37:26.658 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-09-02 13:37:26.658 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:37:26.663 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:37:26.665 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:37:26.665 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:37:26.665 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:37:26.667 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:37:26.667 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:37:26.670 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:26.670 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-09-02 13:37:26.676 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:37:26.680 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:37:26.684 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:37:26.684 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:37:26.687 INFO 13484 --- [ocp-schedules-6,3b2e05d81a2a84ec,5a2a2434610afb54] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:37:26.692 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:37:26.702 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-09-02 13:37:26.710 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-09-02 13:37:26.713 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:26.716 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:26.717 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-09-02 13:37:26.717 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-09-02 13:37:26.722 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-09-02 13:37:26.724 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-09-02 13:37:26.735 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-09-02 13:37:26.741 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:37:26.745 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:37:26.750 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-09-02 13:37:26.755 WARN 13484 --- [scanner-12210645-1,3b2e05d81a2a84ec,7d22b29fc86689c9] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:37:26.760 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:37:26.761 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-09-02 13:37:26.762 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:26.764 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:26.764 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-09-02 13:37:26.764 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-09-02 13:37:26.764 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-09-02 13:37:26.765 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-09-02 13:37:26.768 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:26.771 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-09-02 13:37:26.772 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-09-02 13:37:26.775 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:37:26.777 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-09-02 13:37:26.777 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-09-02 13:37:26.785 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-09-02 13:37:26.798 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:37:26.802 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:26.802 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.025724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:37:26.809 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-09-02 13:37:26.809 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:37:26.817 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:37:26.817 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:37:26.835 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-09-02 13:37:26.837 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.025724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:26.837 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:37:26.840 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-09-02 13:37:26.845 INFO 13484 --- [host-check-host-status4,3dacd66de410e247,0077ae5adbdfe8ac] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-09-02 13:37:26.848 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:37:26.859 INFO 13484 --- [host-check-host-status5,3dacd66de410e247,78ffb69354aaa6ed] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-09-02 13:37:26.861 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:26.861 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27.368133+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:37:26.872 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-09-02 13:37:26.875 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-09-02 13:37:26.880 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27.368133+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:26.880 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:37:26.880 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-09-02 13:37:26.883 INFO 13484 --- [host-check-host-status3,3dacd66de410e247,f093006e4826be8c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-09-02 13:37:26.886 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-09-02 13:37:26.894 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:26.894 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.373150+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:37:26.896 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-09-02 13:37:26.900 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:37:26.936 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:37:26.936 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.373150+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:26.951 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:26.953 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:26.953 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-09-02 13:37:26.957 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster myoceanbase 2025-09-02 13:37:26.962 INFO 13484 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 3, sqlPort: 2881 2025-09-02 13:37:26.966 INFO 13484 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:37:26.968 INFO 13484 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:37:26.974 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-09-02 13:37:26.983 INFO 13484 --- [host-check-host-status1,3dacd66de410e247,eb0cdf4f2bca908a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-09-02 13:37:26.985 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:37:26.984562+08:00, lastProcessTime=2025-09-02T13:37:16.984849+08:00 2025-09-02 13:37:26.987 INFO 13484 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-09-02 13:37:27.010 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-09-02 13:37:27.018 INFO 13484 --- [host-check-host-status2,3dacd66de410e247,c2fc2840d7f774e1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-09-02 13:37:27.021 INFO 13484 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 3, sqlPort: 2881 got ObProcessInfoResult(pid=4794, runPath=/data1/myoceanbase/oceanbase, runUser=root, execute=/data1/myoceanbase/oceanbase/bin/observer, installPath=/data1/myoceanbase/oceanbase) 2025-09-02 13:37:27.034 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:27.035 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:27.036 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-09-02 13:37:27.051 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=myoceanbase, obClusterId=1755130302 2025-09-02 13:37:27.051 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=myoceanbase, obClusterId=1755130302, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"myoceanbase","ObClusterId":1755130302,"ObRegion":"myoceanbase","ObRegionId":1755130302,"ReadonlyRsList":[],"RsList":[{"address":"172.17.172.85:2882","role":"LEADER","sql_port":2881},{"address":"172.17.172.86:2882","role":"FOLLOWER","sql_port":2881},{"address":"172.17.172.84:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1756791174423000}, createTime=2025-08-29T16:02:30+08:00, creator=admin, obVersion=4.3.5.3, packageOperatingSystem=null, loadType=null, attributesJson={"DataDiskPath":"/data1/myoceanbase/oceanbase/store","DiskPathStyle":"OBD","InstallPath":"/data1/myoceanbase/oceanbase","LogDiskPath":"/data1/myoceanbase/oceanbase/store","OperatingSystemUser":"root","RunPath":"/data1/myoceanbase/oceanbase","SqlPort":2881,"SvrPort":2882}, tenantCount=2) 2025-09-02 13:37:27.052 INFO 13484 --- [ocp-schedules-20,0ad4b2692f1472f8,35a62fb2c2616b83] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:37:27.146 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:37:26.984562+08:00 2025-09-02 13:37:27.953 INFO 13484 --- [ocp-schedules-17,08a42cb047289c69,3aa6052bd51a8fe1] c.o.o.s.common.scheduled.PerfSchedules : Collect slow sql for alarm, time=2025-09-02T13:37:27.953+08:00, clusters=[1] 2025-09-02 13:37:28.012 INFO 13484 --- [ocp-schedules-17,08a42cb047289c69,3aa6052bd51a8fe1] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for myoceanbase, save 10 to SqlAggregateData 2025-09-02 13:37:28.266 INFO 13484 --- [ocp-schedules-28,5f405e0722f7f76e,083ae52012397049] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:37:28.323 INFO 13484 --- [prometheus-minute-schedule-1,9143abd1bc9bc414,101c00084e2ec417] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:28.327 INFO 13484 --- [prometheus-minute-schedule-1,9143abd1bc9bc414,101c00084e2ec417] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:28.431 INFO 13484 --- [ocp-async-4,694b32c6b9360970,e114882566e25acb] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:28.969 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:28.970 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:29.094 INFO 13484 --- [ocp-async-10,d28698297530666c,4f7f746fe995d651] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:37:29.744 INFO 13484 --- [host-check-clockdiff1,bd450de42c26563c,940231cae4f9ae50] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791449 0 0 , error= 2025-09-02 13:37:29.829 INFO 13484 --- [ocp-schedules-28,6ebb830efce6690d,d6d4f0ac64876de1] c.o.o.s.c.scheduled.TenantSchedules : Sync all tenant info schedule. 2025-09-02 13:37:29.830 INFO 13484 --- [ocp-schedules-28,6ebb830efce6690d,d6d4f0ac64876de1] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:37:29.872 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-09-02 13:37:29.872 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:29.876 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) ocp_query */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2025-09-02 13:37:29.891 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:29.892 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-09-02 13:37:29.922 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:29.922 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2025-09-02 13:37:29.956 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:29.957 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1] 2025-09-02 13:37:29.961 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:29.962 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2025-09-02 13:37:29.994 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:29.994 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2025-09-02 13:37:30.023 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:30.024 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, SERVICE_NAME, SERVICE_STATUS FROM CDB_OB_SERVICES WHERE TENANT_ID = ?, args: [1002] 2025-09-02 13:37:30.040 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:30.041 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2025-09-02 13:37:30.044 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:30.045 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-09-02 13:37:30.047 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:30.048 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-09-02 13:37:30.051 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:30.052 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2025-09-02 13:37:30.055 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:30.056 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2025-09-02 13:37:30.059 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:30.060 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2025-09-02 13:37:30.062 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2025-09-02 13:37:30.065 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=172.19.62.162 2025-09-02 13:37:30.087 INFO 13484 --- [Sync all tenant information1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:30.089 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:30.089 INFO 13484 --- [Sync all tenant information1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE name = ?, args: [ob_tcp_invited_nodes] 2025-09-02 13:37:30.103 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,172.19.62.162, userWhitelist=% 2025-09-02 13:37:30.103 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2025-09-02 13:37:30.103 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,172.19.62.162, userWhitelist=% 2025-09-02 13:37:30.103 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2025-09-02 13:37:30.106 INFO 13484 --- [Sync all tenant information1,,] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2025-09-02 13:37:30.106 INFO 13484 --- [Sync all tenant information1,,] .o.s.t.b.t.SyncAllTenantInfoShardingTask : [OCP606] sync tenants success, clusterId=1, clusterName=myoceanbase, obClusterId=1755130302 2025-09-02 13:37:30.106 INFO 13484 --- [ocp-schedules-28,6ebb830efce6690d,d6d4f0ac64876de1] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all tenant information executed completed, allCount=1, successCount=1 2025-09-02 13:37:30.431 INFO 13484 --- [ocp-async-9,1ced5d942208fe31,f6ff9055851806e2] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:30.861 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.m.helper.ExporterRequestHelper : Refresh exporter helper 2025-09-02 13:37:30.875 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:37:30.877 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:37:30.899 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:37:30.901 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:37:30.923 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:37:30.924 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:37:30.933 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,4850fb085993ed6b,da097dc0dfc55dd2] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=4850fb085993ed6b, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:37:30.933 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,6dddb5b4e915b75b,8c73fa2876401e2b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=6dddb5b4e915b75b, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:37:30.934 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,6dddb5b4e915b75b,8c73fa2876401e2b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=6dddb5b4e915b75b, duration=4 ms] 2025-09-02 13:37:30.934 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,4850fb085993ed6b,da097dc0dfc55dd2] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=4850fb085993ed6b, duration=4 ms] 2025-09-02 13:37:30.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,69bb3eced79e8d9c,d8ecfc3877382526] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=69bb3eced79e8d9c, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:37:30.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,99d3a857ae1f8903,7a30672708fca81f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=99d3a857ae1f8903, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:37:30.935 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,69bb3eced79e8d9c,d8ecfc3877382526] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=69bb3eced79e8d9c, duration=5 ms] 2025-09-02 13:37:30.937 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-10,99d3a857ae1f8903,7a30672708fca81f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=99d3a857ae1f8903, duration=7 ms] 2025-09-02 13:37:30.948 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:37:30.950 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:37:30.972 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:37:30.974 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:37:30.995 INFO 13484 --- [ocp-schedules-7,1df0e3faf8dcafc9,d2c5a5513d7182c3] c.o.o.m.helper.ExporterRequestHelper : Refresh exporter helper finished, credentialSize=5, cachedSize=5 2025-09-02 13:37:31.323 INFO 13484 --- [prometheus-minute-schedule-1,e204c485ede00fc8,d22dd246465ba82c] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:31.328 INFO 13484 --- [prometheus-minute-schedule-1,e204c485ede00fc8,d22dd246465ba82c] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:31.970 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:31.971 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:32.431 INFO 13484 --- [ocp-async-6,f10dcf31867f9d44,ebc991a32996972c] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:32.770 INFO 13484 --- [ocp-schedules-24,c01f7067761f299d,3dcfbcd8106eb8c9] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:37:32.772 INFO 13484 --- [ocp-schedules-24,c01f7067761f299d,3dcfbcd8106eb8c9] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:37:34.323 INFO 13484 --- [prometheus-minute-schedule-1,7333c2f91974d805,db2d18067f70819a] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:34.328 INFO 13484 --- [prometheus-minute-schedule-1,7333c2f91974d805,db2d18067f70819a] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:34.431 INFO 13484 --- [ocp-async-8,bf2601850adf9696,b2e789684b73749a] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:34.933 INFO 13484 --- [ocp-schedules-13,f50cff780bea06b0,dff492cf39b6fe50] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-09-02 13:37:34.971 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:34.972 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:36.432 INFO 13484 --- [ocp-async-5,9f2144d49c1acaf3,075ecab2b01e1f3a] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:36.863 INFO 13484 --- [ocp-schedules-17,cec497dc9bb05f60,19bc9d18815728ee] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:37:36.892 WARN 13484 --- [scanner-12210766-1,cec497dc9bb05f60,9d442d0a084f4317] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:37:36.899 INFO 13484 --- [alarm-detect-20,cec497dc9bb05f60,8f9a56f218203ae9] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 3016178, data is empty. 2025-09-02 13:37:36.910 INFO 13484 --- [alarm-detect-3,cec497dc9bb05f60,33700c91004ece4a] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 3016178, data is empty. 2025-09-02 13:37:36.913 INFO 13484 --- [alarm-detect-13,cec497dc9bb05f60,92af248efaaeacab] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 3016178, data is empty. 2025-09-02 13:37:36.963 INFO 13484 --- [alarm-detect-10,cec497dc9bb05f60,b54a4e012b17574d] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 3016178, data is empty. 2025-09-02 13:37:36.985 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:37:36.984482+08:00, lastProcessTime=2025-09-02T13:37:26.984562+08:00 2025-09-02 13:37:37.145 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:37:36.984482+08:00 2025-09-02 13:37:37.323 INFO 13484 --- [prometheus-minute-schedule-1,1cd6b320ed796cb5,e066033c906d629d] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:37.328 INFO 13484 --- [prometheus-minute-schedule-1,1cd6b320ed796cb5,e066033c906d629d] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:37.972 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:37.973 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:38.270 INFO 13484 --- [ocp-schedules-13,b9efb7aea54cb425,1b0aff214ac6c0db] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:37:38.432 INFO 13484 --- [ocp-async-2,f28bb6fcb9e6b7af,06889597e4a0bcde] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:39.095 INFO 13484 --- [ocp-async-9,d117d800295e264d,773d7ac73d9b6686] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:37:40.005 INFO 13484 --- [ocp-schedules-3,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=2 2025-09-02 13:37:40.005 INFO 13484 --- [ocp-schedules-21,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=2 2025-09-02 13:37:40.021 INFO 13484 --- [Check obproxy2,3f7fce154e8443fb,26c1e1aeb97a566a] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:37:40.021 INFO 13484 --- [Check obproxy2,3f7fce154e8443fb,26c1e1aeb97a566a] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:37:40.024 INFO 13484 --- [Collect obproxy2,4821d696a4e9c209,325fdc176a690413] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:37:40.024 INFO 13484 --- [Collect obproxy2,4821d696a4e9c209,325fdc176a690413] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:37:40.024 INFO 13484 --- [Check obproxy1,55adcc7f03138470,7092b09ab4d4db35] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:37:40.024 INFO 13484 --- [Check obproxy1,55adcc7f03138470,7092b09ab4d4db35] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:37:40.027 INFO 13484 --- [Collect obproxy1,548bacd045939742,f881aab1e15bccb3] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:37:40.027 INFO 13484 --- [Collect obproxy1,548bacd045939742,f881aab1e15bccb3] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:37:40.050 INFO 13484 --- [Check obproxy2,3f7fce154e8443fb,26c1e1aeb97a566a] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:37:40.051 INFO 13484 --- [Check obproxy1,55adcc7f03138470,7092b09ab4d4db35] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:37:40.053 INFO 13484 --- [Collect obproxy2,4821d696a4e9c209,325fdc176a690413] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:37:40.053 INFO 13484 --- [Check obproxy2,3f7fce154e8443fb,26c1e1aeb97a566a] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-09-02 13:37:40.053 INFO 13484 --- [Check obproxy2,3f7fce154e8443fb,26c1e1aeb97a566a] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 5, need to sync to meta: false 2025-09-02 13:37:40.054 INFO 13484 --- [Check obproxy1,55adcc7f03138470,7092b09ab4d4db35] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-09-02 13:37:40.054 INFO 13484 --- [Check obproxy1,55adcc7f03138470,7092b09ab4d4db35] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 6, need to sync to meta: false 2025-09-02 13:37:40.055 INFO 13484 --- [Collect obproxy1,548bacd045939742,f881aab1e15bccb3] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:37:40.056 INFO 13484 --- [Collect obproxy2,4821d696a4e9c209,325fdc176a690413] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:37:40.058 INFO 13484 --- [Collect obproxy1,548bacd045939742,f881aab1e15bccb3] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:37:40.059 INFO 13484 --- [Collect obproxy2,4821d696a4e9c209,325fdc176a690413] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:37:40.060 INFO 13484 --- [Collect obproxy1,548bacd045939742,f881aab1e15bccb3] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:37:40.323 INFO 13484 --- [prometheus-minute-schedule-1,693b5c52d3517ba4,ed9d736a4285702f] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:40.328 INFO 13484 --- [prometheus-minute-schedule-1,693b5c52d3517ba4,ed9d736a4285702f] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:40.432 INFO 13484 --- [ocp-async-8,55aa0c32b01fd210,70623f1b550c1672] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:40.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,941a73aa097221f0,d73ec9a6fa42cef7] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=941a73aa097221f0, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:37:40.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,de2b2e3006db1167,814345bd6bdab14f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=de2b2e3006db1167, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:37:40.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,f163d245858d398b,498ff057f736ac09] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=f163d245858d398b, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:37:40.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,49763a29bb8998c1,f87f483f6e97bb39] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=49763a29bb8998c1, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:37:40.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,49763a29bb8998c1,f87f483f6e97bb39] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=49763a29bb8998c1, duration=5 ms] 2025-09-02 13:37:40.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,f163d245858d398b,498ff057f736ac09] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=f163d245858d398b, duration=5 ms] 2025-09-02 13:37:40.923 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,941a73aa097221f0,d73ec9a6fa42cef7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=941a73aa097221f0, duration=6 ms] 2025-09-02 13:37:40.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,de2b2e3006db1167,814345bd6bdab14f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=de2b2e3006db1167, duration=7 ms] 2025-09-02 13:37:40.973 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:40.973 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:41.323 INFO 13484 --- [prometheus-cache-monitor1,b1c1af188bbb05e7,8f418050ca8c8695] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":10712,"minuteWriteQueue":0,"minuteReadCacheSize":4820,"secondWriteCacheSize":2678,"seriesIdKeySize":5749,"secondWriteQueue":0,"minuteWriteCacheSize":4806} 2025-09-02 13:37:41.324 INFO 13484 --- [prometheus-cache-monitor1,b1c1af188bbb05e7,8f418050ca8c8695] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{ob_sysstat_40012=15, ob_server_resource_memory_assigned_bytes=10, ob_sysstat_40011=15, ob_sysstat_40010=15, ob_unit_num=10, ob_database_disk_required_size=29, go_memstats_mspan_sys_bytes=16, ob_sysstat_30213=15, go_memstats_stack_sys_bytes=16, ob_sysstat_30212=15, ob_sysstat_30217=15, ob_sysstat_30216=15, ob_tenant_request_queue_total_size=16, jvm_gc_memory_allocated_bytes_total=2, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=10, ob_sysstat_40019=15, ob_sysstat_40018=15, processor_process_metrics_total=20, jvm_gc_max_data_size_bytes=2, ob_sysstat_40027=15, ob_sysstat_40026=15, ob_sysstat_40025=15, go_memstats_frees_total=16, host_available=15, ob_sysstat_30220=15, ob_sysstat_30222=15, ob_sysstat_30221=15, mysql_output_write_sql_failed_count=19, ob_database_disk_data_size=28, ob_tenant_memstore_freeze_cnt=12, ob_tenant_unit_min_memory=21, ob_sysstat_40028=15, ob_sysstat_60087=15, ob_memstore_total_bytes=16, jvm_threads_states_threads=8, process_resident_memory_bytes=16, ob_cache_size_bytes=33, ob_tenant_unit_max_memory=21, input_collect_metrics_total=24, ob_sysstat_50039=15, ob_sysstat_60083=15, ob_sysstat_50038=15, node_disk_io_time_weighted_seconds_total=27, process_open_fds=16, ob_sysstat_50037=15, process_memory_rss_bytes=29, ob_sysstat_50030=15, ob_server_resource_cpu_assigned_percent=10, sql_audit_input_collect_lost_total=12, go_memstats_heap_inuse_bytes=16, ob_sysstat_40118=15, ob_sysstat_40117=15, node_memory_memfree_bytes=20, ob_sysstat_40116=15, ob_sysstat_40005=15, process_exists=39, ob_sysstat_40004=15, ob_sysstat_40003=15, ob_sysstat_40002=15, go_gc_duration_seconds=21, ob_server_resource_disk_bytes=10, ob_sysstat_40001=15, ob_sysstat_40000=15, mysql_output_write_sql_time=19, ob_sysstat_30200=15, odp_current_session=17, ob_sysstat_30201=15, ob_sysstat_30204=15, ob_sysstat_30205=15, ob_sysstat_30208=15, process_thread_count=29, ob_sysstat_30209=15, ob_sysstat_50040=15, ob_sysstat_40009=15, ob_disk_free_bytes=10, ob_sysstat_40008=15, node_filesystem_avail_bytes=49, 2025-09-02 13:37:42.432 INFO 13484 --- [ocp-async-5,2185b401315680fa,63c565c7fb8af057] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:42.780 INFO 13484 --- [ocp-schedules-30,3046c8531c4a66f3,34bbf1b24b6e72ac] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:37:42.782 INFO 13484 --- [ocp-schedules-30,3046c8531c4a66f3,34bbf1b24b6e72ac] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:37:43.323 INFO 13484 --- [prometheus-minute-schedule-1,7e439fd40f34e80a,b6875923f74f5866] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:43.327 INFO 13484 --- [prometheus-minute-schedule-1,7e439fd40f34e80a,b6875923f74f5866] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:43.781 INFO 13484 --- [ocp-schedules-9,00cd3739f975019f,7db71bf26c9a3447] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-09-02 13:37:43.788 INFO 13484 --- [ocp-schedules-9,00cd3739f975019f,7db71bf26c9a3447] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:37:43.788 INFO 13484 --- [ocp-schedules-9,00cd3739f975019f,7db71bf26c9a3447] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:37:43.974 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:43.974 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:44.432 INFO 13484 --- [ocp-async-2,f49329d93adabf60,3f0cb602fda0d5ff] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:45.101 INFO 13484 --- [prometheus-second-schedule-1,bff308931fb1d010,f4fb42aa83dfa8b4] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=15, collectTaskCount=15 2025-09-02 13:37:46.323 INFO 13484 --- [prometheus-minute-schedule-1,32fb6bbfabdf7bd8,495726c8c96cfe30] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:46.328 INFO 13484 --- [prometheus-minute-schedule-1,32fb6bbfabdf7bd8,495726c8c96cfe30] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:46.433 INFO 13484 --- [ocp-async-10,6800ad4a08bf3da2,416fb4e4e93cfee7] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:46.975 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:46.976 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:46.981 INFO 13484 --- [ocp-schedules-25,378f98275281d590,c80330dceb372e3f] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:37:46.989 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:37:46.988316+08:00, lastProcessTime=2025-09-02T13:37:36.984482+08:00 2025-09-02 13:37:47.041 WARN 13484 --- [scanner-12210939-1,378f98275281d590,c7892f4cee317d36] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:37:47.239 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:37:46.988316+08:00 2025-09-02 13:37:48.275 INFO 13484 --- [ocp-schedules-30,fe8f91db48c2d627,4c87116c6a602358] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:37:48.433 INFO 13484 --- [ocp-async-7,5e0e9d66cd87e475,2743343f184d3138] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:48.924 INFO 13484 --- [ocp-schedules-10,e45f6b4b6f017a76,e303d32a1232cfa6] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster zone cgroup value. 2025-09-02 13:37:48.926 INFO 13484 --- [ocp-schedules-10,e45f6b4b6f017a76,e303d32a1232cfa6] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:37:48.951 INFO 13484 --- [Check cluster cgroup value.1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:48.952 INFO 13484 --- [Check cluster cgroup value.1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [enable_cgroup] 2025-09-02 13:37:48.964 INFO 13484 --- [ocp-schedules-10,e45f6b4b6f017a76,e303d32a1232cfa6] .o.o.s.t.b.c.AbstractShardingClusterTask : Check cluster cgroup value. executed completed, allCount=1, successCount=1 2025-09-02 13:37:49.095 INFO 13484 --- [ocp-async-4,377dfb4ff1dd5c3b,7a703e419ef0905b] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:37:49.323 INFO 13484 --- [prometheus-minute-schedule-1,36f98b10705c0d24,df3809dafafae4a7] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:49.328 INFO 13484 --- [prometheus-minute-schedule-1,36f98b10705c0d24,df3809dafafae4a7] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:49.976 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:49.977 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:50.433 INFO 13484 --- [ocp-async-2,c83032fb89b41017,7887f9973b37546c] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:50.919 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,bc397e74555636b2,90c8da0397fcd607] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=bc397e74555636b2, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:37:50.919 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,0ea0693900dbe62d,45cd6368e0210cfe] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=0ea0693900dbe62d, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:37:50.919 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,2d039d106bece7f8,8f5942eec13fa9cc] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=2d039d106bece7f8, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:37:50.919 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,50ae0d883f400a35,64375261390c53da] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=50ae0d883f400a35, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:37:50.919 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,bc397e74555636b2,90c8da0397fcd607] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=bc397e74555636b2, duration=3 ms] 2025-09-02 13:37:50.920 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,2d039d106bece7f8,8f5942eec13fa9cc] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=2d039d106bece7f8, duration=4 ms] 2025-09-02 13:37:50.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,50ae0d883f400a35,64375261390c53da] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=50ae0d883f400a35, duration=6 ms] 2025-09-02 13:37:50.923 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-9,0ea0693900dbe62d,45cd6368e0210cfe] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=0ea0693900dbe62d, duration=7 ms] 2025-09-02 13:37:52.323 INFO 13484 --- [prometheus-minute-schedule-1,16615bfec2d137f3,d97d0fb86271dc5c] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:52.328 INFO 13484 --- [prometheus-minute-schedule-1,16615bfec2d137f3,d97d0fb86271dc5c] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:52.433 INFO 13484 --- [ocp-async-10,0188bd44033827d9,bf09a89e12c39354] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:52.788 INFO 13484 --- [ocp-schedules-28,5505507b82285cf1,e416f98bca632693] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:37:52.790 INFO 13484 --- [ocp-schedules-28,5505507b82285cf1,e416f98bca632693] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:37:52.977 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:52.978 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:54.433 INFO 13484 --- [ocp-async-7,7bb86d1a5c04b059,8ddf7b7ff7002254] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:55.323 INFO 13484 --- [prometheus-minute-schedule-1,7bbd08fb1f5f02c8,cac897f326fc72f5] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:55.328 INFO 13484 --- [prometheus-minute-schedule-1,7bbd08fb1f5f02c8,cac897f326fc72f5] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:55.978 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:55.979 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:56.434 INFO 13484 --- [ocp-async-9,49c559d0a4bda439,6c28ba8120360a27] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:56.539 INFO 13484 --- [ocp-schedules-21,9a23cd74cded6678,6c9dd45b62c5e1c9] c.o.o.s.c.scheduled.TenantSchedules : Check all standby tenant 2025-09-02 13:37:56.552 INFO 13484 --- [ocp-schedules-4,39c875d0697d2263,593dd10d432b9879] c.o.o.s.common.scheduled.HostSchedules : Check host status schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:37:56.552 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=172.19.62.154, originalStatus=ONLINE 2025-09-02 13:37:56.552 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=172.19.62.156, originalStatus=ONLINE 2025-09-02 13:37:56.552 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=172.17.172.84, originalStatus=ONLINE 2025-09-02 13:37:56.556 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:37:56.556 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:37:56.558 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:37:56.558 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:37:56.558 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=172.17.172.86, originalStatus=ONLINE 2025-09-02 13:37:56.558 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=172.17.172.85, originalStatus=ONLINE 2025-09-02 13:37:56.558 INFO 13484 --- [ocp-schedules-23,fa94fbaa8960e0ea,764eb649a6588598] c.o.o.s.c.scheduled.ClusterSchedules : Check cluster status schedule. 2025-09-02 13:37:56.563 INFO 13484 --- [ocp-schedules-23,fa94fbaa8960e0ea,764eb649a6588598] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:37:56.565 INFO 13484 --- [ocp-schedules-29,aa8d535fdebbc925,fd84eb70f7d3a5cd] c.o.o.s.c.scheduled.ClusterSchedules : Check standby cluster info schedule. 2025-09-02 13:37:56.565 INFO 13484 --- [ocp-schedules-8,b549b24ccb368eaf,b3347129f5031555] c.o.o.s.c.scheduled.ClusterSchedules : Sync cluster info schedule. 2025-09-02 13:37:56.567 INFO 13484 --- [ocp-schedules-8,b549b24ccb368eaf,b3347129f5031555] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:37:56.569 INFO 13484 --- [ocp-schedules-30,3a6a0a5b7b64634b,13502c0faaa214ee] c.o.o.s.common.scheduled.HostSchedules : Check host agent status schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:37:56.569 INFO 13484 --- [host-check-agent-status1,3a6a0a5b7b64634b,0efeacc1094a44b7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=172.19.62.154, originalStatus=ONLINE 2025-09-02 13:37:56.569 INFO 13484 --- [host-check-agent-status2,3a6a0a5b7b64634b,cb7f7c6044d54e1f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=172.19.62.156, originalStatus=ONLINE 2025-09-02 13:37:56.570 INFO 13484 --- [host-check-agent-status3,3a6a0a5b7b64634b,ca5ca41f75d4c11e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=172.17.172.84, originalStatus=ONLINE 2025-09-02 13:37:56.570 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:37:56.570 INFO 13484 --- [host-check-agent-status4,3a6a0a5b7b64634b,786b7f76f60818e8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=172.17.172.86, originalStatus=ONLINE 2025-09-02 13:37:56.571 INFO 13484 --- [host-check-agent-status5,3a6a0a5b7b64634b,d685ea1a39793d4b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=172.17.172.85, originalStatus=ONLINE 2025-09-02 13:37:56.571 INFO 13484 --- [ocp-schedules-25,9c5361118f9b66f2,23ef4ed5fed35534] c.o.o.s.common.scheduled.HostSchedules : Check host clockdiff schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:37:56.573 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:37:56.573 INFO 13484 --- [ocp-schedules-29,aa8d535fdebbc925,fd84eb70f7d3a5cd] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:37:56.575 INFO 13484 --- [host-check-clockdiff1,9c5361118f9b66f2,6082afc3c554e1d3] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.19.62.154], timeoutMillis=20000 2025-09-02 13:37:56.576 INFO 13484 --- [host-check-clockdiff2,9c5361118f9b66f2,c249057026f93682] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.19.62.156], timeoutMillis=20000 2025-09-02 13:37:56.577 INFO 13484 --- [host-check-clockdiff3,9c5361118f9b66f2,fb240618c78cda87] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.84], timeoutMillis=20000 2025-09-02 13:37:56.578 INFO 13484 --- [ocp-schedules-1,62effaa92226d993,246894538e2dfd3f] c.o.o.s.c.scheduled.ClusterSchedules : Check all arbitration services 2025-09-02 13:37:56.578 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:37:56.578 INFO 13484 --- [host-check-agent-status2,3a6a0a5b7b64634b,cb7f7c6044d54e1f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:37:56.579 INFO 13484 --- [host-check-clockdiff4,9c5361118f9b66f2,f70360d57c708302] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.86], timeoutMillis=20000 2025-09-02 13:37:56.579 INFO 13484 --- [ocp-schedules-29,aa8d535fdebbc925,fd84eb70f7d3a5cd] .o.o.s.t.b.c.AbstractShardingClusterTask : Check all standby clusters sync delay time executed completed, allCount=1, successCount=1 2025-09-02 13:37:56.579 INFO 13484 --- [host-check-agent-status1,3a6a0a5b7b64634b,0efeacc1094a44b7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:37:56.579 INFO 13484 --- [host-check-clockdiff5,9c5361118f9b66f2,0e0fd313ce4bdb54] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[172.17.172.85], timeoutMillis=20000 2025-09-02 13:37:56.579 INFO 13484 --- [host-check-agent-status5,3a6a0a5b7b64634b,d685ea1a39793d4b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:37:56.580 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:37:56.580 INFO 13484 --- [host-check-agent-status2,3a6a0a5b7b64634b,cb7f7c6044d54e1f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:37:56.581 INFO 13484 --- [host-check-agent-status5,3a6a0a5b7b64634b,d685ea1a39793d4b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:37:56.581 INFO 13484 --- [host-check-agent-status1,3a6a0a5b7b64634b,0efeacc1094a44b7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:37:56.598 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:37:56.598 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62888/api/v1/time, params:null 2025-09-02 13:37:56.600 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:37:56.601 INFO 13484 --- [host-check-agent-status4,3a6a0a5b7b64634b,786b7f76f60818e8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:37:56.601 INFO 13484 --- [host-check-agent-status3,3a6a0a5b7b64634b,ca5ca41f75d4c11e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:37:56.601 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=172.19.62.156, timestamp=2025-09-02T13:37:56.599846160+08:00 2025-09-02 13:37:56.602 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2025-09-02T13:37:56.601+08:00 2025-09-02 13:37:56.603 INFO 13484 --- [host-check-agent-status3,3a6a0a5b7b64634b,ca5ca41f75d4c11e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:37:56.611 INFO 13484 --- [host-check-agent-status4,3a6a0a5b7b64634b,786b7f76f60818e8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:37:56.612 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2025-09-02T13:37:56.601+08:00 and status AVAILABLE 2025-09-02 13:37:56.612 INFO 13484 --- [host-check-clockdiff2,9c5361118f9b66f2,c249057026f93682] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791476 -1 -1 , error= 2025-09-02 13:37:56.612 INFO 13484 --- [host-check-clockdiff4,9c5361118f9b66f2,f70360d57c708302] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791476 0 0 , error= 2025-09-02 13:37:56.621 INFO 13484 --- [host-check-clockdiff3,9c5361118f9b66f2,fb240618c78cda87] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791476 0 0 , error= 2025-09-02 13:37:56.622 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62888/api/v1/time, params:null 2025-09-02 13:37:56.625 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2025-09-02 13:37:56.628 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=172.17.172.85, timestamp=2025-09-02T13:37:56.626358278+08:00 2025-09-02 13:37:56.629 INFO 13484 --- [host-check-clockdiff5,9c5361118f9b66f2,0e0fd313ce4bdb54] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791476 -1 -1 , error= 2025-09-02 13:37:56.643 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2025-09-02T13:37:56.628+08:00 2025-09-02 13:37:56.646 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2025-09-02T13:37:56.628+08:00 and status AVAILABLE 2025-09-02 13:37:56.651 INFO 13484 --- [host-check-clockdiff1,9c5361118f9b66f2,6082afc3c554e1d3] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, exitCode=0, out=1756791476 0 0 , error= 2025-09-02 13:37:56.651 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:37:56.651 INFO 13484 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.653 INFO 13484 --- [Check status for all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-09-02 13:37:56.653 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:37:56.654 INFO 13484 --- [host-check-agent-status5,3a6a0a5b7b64634b,d685ea1a39793d4b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62888/api/v1/agent/status, params:null 2025-09-02 13:37:56.654 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2025-09-02 13:37:56.656 INFO 13484 --- [host-check-agent-status1,3a6a0a5b7b64634b,0efeacc1094a44b7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62888/api/v1/agent/status, params:null 2025-09-02 13:37:56.656 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.657 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-09-02 13:37:56.657 INFO 13484 --- [Check status for all clusters1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.3.5.3 2025-09-02 13:37:56.657 INFO 13484 --- [ocp-schedules-23,fa94fbaa8960e0ea,764eb649a6588598] .o.o.s.t.b.c.AbstractShardingClusterTask : Check status for all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:37:56.658 INFO 13484 --- [host-check-agent-status5,3a6a0a5b7b64634b,d685ea1a39793d4b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.85 2025-09-02 13:37:56.658 INFO 13484 --- [host-check-agent-status5,3a6a0a5b7b64634b,d685ea1a39793d4b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.85 2025-09-02 13:37:56.658 INFO 13484 --- [host-check-agent-status5,3a6a0a5b7b64634b,d685ea1a39793d4b] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.85 2025-09-02 13:37:56.659 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62888/api/v1/time, params:null 2025-09-02 13:37:56.660 INFO 13484 --- [host-check-agent-status1,3a6a0a5b7b64634b,0efeacc1094a44b7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.19.62.154 2025-09-02 13:37:56.660 INFO 13484 --- [host-check-agent-status1,3a6a0a5b7b64634b,0efeacc1094a44b7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.19.62.154 2025-09-02 13:37:56.660 INFO 13484 --- [host-check-agent-status1,3a6a0a5b7b64634b,0efeacc1094a44b7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.19.62.154 2025-09-02 13:37:56.664 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:37:56.665 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=172.19.62.154, timestamp=2025-09-02T13:37:56.664937651+08:00 2025-09-02 13:37:56.666 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2025-09-02T13:37:56.665+08:00 2025-09-02 13:37:56.666 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62888/api/v1/time, params:null 2025-09-02 13:37:56.666 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:37:56.667 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62888/api/v1/time, params:null 2025-09-02 13:37:56.668 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2025-09-02T13:37:56.665+08:00 and status AVAILABLE 2025-09-02 13:37:56.668 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=172.17.172.84, timestamp=2025-09-02T13:37:56.667702848+08:00 2025-09-02 13:37:56.668 INFO 13484 --- [host-check-agent-status2,3a6a0a5b7b64634b,cb7f7c6044d54e1f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62888/api/v1/agent/status, params:null 2025-09-02 13:37:56.669 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2025-09-02T13:37:56.668+08:00 2025-09-02 13:37:56.670 INFO 13484 --- [host-check-agent-status3,3a6a0a5b7b64634b,ca5ca41f75d4c11e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62888/api/v1/agent/status, params:null 2025-09-02 13:37:56.671 INFO 13484 --- [host-check-agent-status2,3a6a0a5b7b64634b,cb7f7c6044d54e1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.19.62.156 2025-09-02 13:37:56.671 INFO 13484 --- [host-check-agent-status2,3a6a0a5b7b64634b,cb7f7c6044d54e1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.19.62.156 2025-09-02 13:37:56.671 INFO 13484 --- [host-check-agent-status2,3a6a0a5b7b64634b,cb7f7c6044d54e1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.19.62.156 2025-09-02 13:37:56.672 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=172.17.172.86, timestamp=2025-09-02T13:37:56.669696958+08:00 2025-09-02 13:37:56.672 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2025-09-02T13:37:56.668+08:00 and status AVAILABLE 2025-09-02 13:37:56.672 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2025-09-02T13:37:56.672+08:00 2025-09-02 13:37:56.673 INFO 13484 --- [host-check-agent-status3,3a6a0a5b7b64634b,ca5ca41f75d4c11e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.84 2025-09-02 13:37:56.673 INFO 13484 --- [host-check-agent-status3,3a6a0a5b7b64634b,ca5ca41f75d4c11e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.84 2025-09-02 13:37:56.673 INFO 13484 --- [host-check-agent-status3,3a6a0a5b7b64634b,ca5ca41f75d4c11e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.84 2025-09-02 13:37:56.675 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2025-09-02T13:37:56.672+08:00 and status AVAILABLE 2025-09-02 13:37:56.676 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:37:56.677 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2025-09-02 13:37:56.681 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2025-09-02 13:37:56.681 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2025-09-02 13:37:56.687 INFO 13484 --- [host-check-agent-status4,3a6a0a5b7b64634b,786b7f76f60818e8] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62888/api/v1/agent/status, params:null 2025-09-02 13:37:56.688 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:37:56.690 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:37:56.690 INFO 13484 --- [host-check-agent-status4,3a6a0a5b7b64634b,786b7f76f60818e8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=172.17.172.86 2025-09-02 13:37:56.690 INFO 13484 --- [host-check-agent-status4,3a6a0a5b7b64634b,786b7f76f60818e8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=172.17.172.86 2025-09-02 13:37:56.690 INFO 13484 --- [host-check-agent-status4,3a6a0a5b7b64634b,786b7f76f60818e8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=172.17.172.86 2025-09-02 13:37:56.693 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:37:56.693 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:37:56.694 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:37:56.694 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:37:56.697 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:37:56.706 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:56.707 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.708 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2025-09-02 13:37:56.713 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:37:56.713 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:37:56.718 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2025-09-02 13:37:56.720 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=13 2025-09-02 13:37:56.729 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=13 2025-09-02 13:37:56.731 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=7 2025-09-02 13:37:56.741 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=8 2025-09-02 13:37:56.747 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:37:56.748 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:56.748 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=2 2025-09-02 13:37:56.749 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=7 2025-09-02 13:37:56.749 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.750 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2025-09-02 13:37:56.752 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=8 2025-09-02 13:37:56.753 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.756 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2025-09-02 13:37:56.757 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=2 2025-09-02 13:37:56.759 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:37:56.764 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=BackupAlarmCheckSchedule, method=run 2025-09-02 13:37:56.765 INFO 13484 --- [ocp-schedules-13,8e115bd9035230aa,eb502a1ff808ee48] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2025-09-02 13:37:56.770 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:37:56.770 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=14 2025-09-02 13:37:56.775 INFO 13484 --- [ocp-schedules-13,8e115bd9035230aa,eb502a1ff808ee48] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2025-09-02 13:37:56.775 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:37:56.775 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.b.i.alarm.BackupAlarmService : Backup alarm check schedule. 2025-09-02 13:37:56.775 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:56.775 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.025724+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:37:56.779 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=14 2025-09-02 13:37:56.779 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:37:56.784 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1 2025-09-02 13:37:56.793 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=10 2025-09-02 13:37:56.793 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=172.17.172.86, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.025724+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:56.793 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:37:56.793 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=9 2025-09-02 13:37:56.794 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1 2025-09-02 13:37:56.800 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.85:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:37:56.803 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=9 2025-09-02 13:37:56.804 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=10 2025-09-02 13:37:56.809 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:56.809 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27.368133+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:37:56.817 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2025-09-02 13:37:56.818 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.819 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:37:56.820 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.821 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:37:56.824 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=15 2025-09-02 13:37:56.824 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.86:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:37:56.828 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:37:56.829 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=172.17.172.85, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=true, hostId=5, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:27.368133+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:56.829 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2025-09-02 13:37:56.833 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.833 INFO 13484 --- [host-check-host-status5,39c875d0697d2263,c51773c016948e28] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=15 2025-09-02 13:37:56.841 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=11 2025-09-02 13:37:56.845 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:56.845 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.373150+08:00, stopTime=1970-01-01T08:00+08:00} 2025-09-02 13:37:56.847 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2025-09-02 13:37:56.851 INFO 13484 --- [host-check-host-status4,39c875d0697d2263,6402ebbbfdeb20bc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=11 2025-09-02 13:37:56.852 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=12 2025-09-02 13:37:56.854 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=3 2025-09-02 13:37:56.862 INFO 13484 --- [host-check-host-status3,39c875d0697d2263,3187a6fc3148bd9f] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=12 2025-09-02 13:37:56.865 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=172.17.172.84, port=2882, sqlPort=2881, installPath=/data1/myoceanbase/oceanbase, runPath=/data1/myoceanbase/oceanbase, runUser=root, version=4.3.5.3_103000092025080818-e8da5f0afb288ed0add0613740c6ccf2a3c6830b(Aug 8 2025 18:44:31), withRootserver=false, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2025-08-14T08:12:28.373150+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2025-09-02 13:37:56.866 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=3 2025-09-02 13:37:56.880 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:56.882 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.883 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2025-09-02 13:37:56.885 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=4 2025-09-02 13:37:56.886 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.156:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:37:56.887 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster myoceanbase 2025-09-02 13:37:56.891 INFO 13484 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 3, sqlPort: 2881 2025-09-02 13:37:56.896 INFO 13484 --- [Sync all clusters1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:37:56.897 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=4 2025-09-02 13:37:56.898 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:56.898 INFO 13484 --- [Sync all clusters1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:37:56.902 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.902 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:37:56.904 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.904 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:37:56.915 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.918 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.19.62.154:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2025-09-02 13:37:56.920 INFO 13484 --- [Sync all clusters1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://172.17.172.84:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2025-09-02 13:37:56.940 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2025-09-02 13:37:56.946 INFO 13484 --- [Sync all clusters1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 3, sqlPort: 2881 got ObProcessInfoResult(pid=4794, runPath=/data1/myoceanbase/oceanbase, runUser=root, execute=/data1/myoceanbase/oceanbase/bin/observer, installPath=/data1/myoceanbase/oceanbase) 2025-09-02 13:37:56.958 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=5 2025-09-02 13:37:56.963 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:56.964 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:56.965 INFO 13484 --- [Sync all clusters1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2025-09-02 13:37:56.968 INFO 13484 --- [host-check-host-status2,39c875d0697d2263,c071bf4812d5dfa9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=5 2025-09-02 13:37:56.972 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2025-09-02 13:37:56.983 INFO 13484 --- [Sync all clusters1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=myoceanbase, obClusterId=1755130302 2025-09-02 13:37:56.983 INFO 13484 --- [Sync all clusters1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=myoceanbase, obClusterId=1755130302, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, origin=TAKEN_OVER, syncStatus=VALID, rootserverJson={"ObCluster":"myoceanbase","ObClusterId":1755130302,"ObRegion":"myoceanbase","ObRegionId":1755130302,"ReadonlyRsList":[],"RsList":[{"address":"172.17.172.85:2882","role":"LEADER","sql_port":2881},{"address":"172.17.172.86:2882","role":"FOLLOWER","sql_port":2881},{"address":"172.17.172.84:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1756791174423000}, createTime=2025-08-29T16:02:30+08:00, creator=admin, obVersion=4.3.5.3, packageOperatingSystem=null, loadType=null, attributesJson={"DataDiskPath":"/data1/myoceanbase/oceanbase/store","DiskPathStyle":"OBD","InstallPath":"/data1/myoceanbase/oceanbase","LogDiskPath":"/data1/myoceanbase/oceanbase/store","OperatingSystemUser":"root","RunPath":"/data1/myoceanbase/oceanbase","SqlPort":2881,"SvrPort":2882}, tenantCount=2) 2025-09-02 13:37:56.983 INFO 13484 --- [ocp-schedules-8,b549b24ccb368eaf,b3347129f5031555] .o.o.s.t.b.c.AbstractShardingClusterTask : Sync all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:37:56.985 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:37:56.984024+08:00, lastProcessTime=2025-09-02T13:37:46.988316+08:00 2025-09-02 13:37:56.988 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=6 2025-09-02 13:37:56.998 INFO 13484 --- [host-check-host-status1,39c875d0697d2263,d3ef25a64b690b24] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=6 2025-09-02 13:37:57.004 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:57.006 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:57.006 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:37:57.011 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:57.012 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:37:57.025 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:57.030 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [600] 2025-09-02 13:37:57.056 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:57.058 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:57.058 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2025-09-02 13:37:57.060 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2025-09-02 13:37:57.076 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.b.i.alarm.BackupAlarmService : clusterId=1, minDelaySeconds=600, delayTasks=[] 2025-09-02 13:37:57.103 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:57.105 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:57.105 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:37:57.107 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:57.108 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:37:57.119 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:57.123 INFO 13484 --- [ocp-schedules-20,226e289fb09a97f5,4ed89684f2057418] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:37:57.133 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [60] 2025-09-02 13:37:57.153 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:37:56.984024+08:00 2025-09-02 13:37:57.158 WARN 13484 --- [scanner-12211068-1,226e289fb09a97f5,2006915ab1197de8] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:37:57.168 INFO 13484 --- [alarm-detect-15,226e289fb09a97f5,f2fd3ffe092fd1a9] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 3016188, data is empty. 2025-09-02 13:37:57.179 INFO 13484 --- [alarm-detect-15,226e289fb09a97f5,61aeabaefdfbb0c5] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 3016188, data is empty. 2025-09-02 13:37:57.179 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=60, timeoutTasks=[] 2025-09-02 13:37:57.180 INFO 13484 --- [alarm-detect-19,226e289fb09a97f5,f802a0290c2c2e68] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 3016188, data is empty. 2025-09-02 13:37:57.217 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=myoceanbase, tenantName=sys, dbUser=ocp_monitor 2025-09-02 13:37:57.219 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:57.220 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE ? 2025-09-02 13:37:57.222 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:57.222 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, name, value, gmt_create, gmt_modified FROM __all_virtual_sys_variable WHERE name = ? 2025-09-02 13:37:57.228 INFO 13484 --- [alarm-detect-9,226e289fb09a97f5,7c3e954227985acb] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 3016188, data is empty. 2025-09-02 13:37:57.233 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:37:57.283 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ * FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT job_id, incarnation, job.tenant_id AS tenant_id, job.backup_set_id AS backup_set_id, backup_type, path AS backup_dest, start_timestamp AS start_time, end_timestamp AS end_time, now(6) AS check_time, status, comment, description, 'TENANT' AS backup_level, file.min_restore_scn_display AS snapshot_version_time, output_bytes FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id, min_restore_scn_display, output_bytes FROM CDB_OB_BACKUP_SET_FILES) file ON job.tenant_id = file.tid AND job.backup_set_id = file.backup_set_id WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2025-09-02 13:37:57.317 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2025-09-02 13:37:57.332 INFO 13484 --- [ocp-schedules-2,044b6a2a18afbd40,f1314bc278f7a0de] c.o.o.b.i.alarm.BackupAlarmService : cluster=myoceanbase does not enable secondary backup 2025-09-02 13:37:57.538 INFO 13484 --- [print-time-series-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Async compute queue max size = 0, current size = 0. 2025-09-02 13:37:57.544 INFO 13484 --- [ocp-schedules-1,10b163e9e6c96e8c,1d93a750eeeb44a0] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":10712,"minuteWriteQueue":0,"minuteReadCacheSize":4820,"secondWriteCacheSize":2678,"seriesIdKeySize":5749,"secondWriteQueue":0,"minuteWriteCacheSize":4806} 2025-09-02 13:37:57.544 INFO 13484 --- [ocp-schedules-1,10b163e9e6c96e8c,1d93a750eeeb44a0] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{ob_sysstat_40012=15, ob_server_resource_memory_assigned_bytes=10, ob_sysstat_40011=15, ob_sysstat_40010=15, ob_unit_num=10, ob_database_disk_required_size=29, go_memstats_mspan_sys_bytes=16, ob_sysstat_30213=15, go_memstats_stack_sys_bytes=16, ob_sysstat_30212=15, ob_sysstat_30217=15, ob_sysstat_30216=15, ob_tenant_request_queue_total_size=16, jvm_gc_memory_allocated_bytes_total=2, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=10, ob_sysstat_40019=15, ob_sysstat_40018=15, processor_process_metrics_total=20, jvm_gc_max_data_size_bytes=2, ob_sysstat_40027=15, ob_sysstat_40026=15, ob_sysstat_40025=15, go_memstats_frees_total=16, host_available=15, ob_sysstat_30220=15, ob_sysstat_30222=15, ob_sysstat_30221=15, mysql_output_write_sql_failed_count=19, ob_database_disk_data_size=28, ob_tenant_memstore_freeze_cnt=12, ob_tenant_unit_min_memory=21, ob_sysstat_40028=15, ob_sysstat_60087=15, ob_memstore_total_bytes=16, jvm_threads_states_threads=8, process_resident_memory_bytes=16, ob_cache_size_bytes=33, ob_tenant_unit_max_memory=21, input_collect_metrics_total=24, ob_sysstat_50039=15, ob_sysstat_60083=15, ob_sysstat_50038=15, node_disk_io_time_weighted_seconds_total=27, process_open_fds=16, ob_sysstat_50037=15, process_memory_rss_bytes=29, ob_sysstat_50030=15, ob_server_resource_cpu_assigned_percent=10, sql_audit_input_collect_lost_total=12, go_memstats_heap_inuse_bytes=16, ob_sysstat_40118=15, ob_sysstat_40117=15, node_memory_memfree_bytes=20, ob_sysstat_40116=15, ob_sysstat_40005=15, process_exists=39, ob_sysstat_40004=15, ob_sysstat_40003=15, ob_sysstat_40002=15, go_gc_duration_seconds=21, ob_server_resource_disk_bytes=10, ob_sysstat_40001=15, ob_sysstat_40000=15, mysql_output_write_sql_time=19, ob_sysstat_30200=15, odp_current_session=17, ob_sysstat_30201=15, ob_sysstat_30204=15, ob_sysstat_30205=15, ob_sysstat_30208=15, process_thread_count=29, ob_sysstat_30209=15, ob_sysstat_50040=15, ob_sysstat_40009=15, ob_disk_free_bytes=10, ob_sysstat_40008=15, node_filesystem_avail_bytes=49, 2025-09-02 13:37:57.591 INFO 13484 --- [ocp-schedules-6,a144c7413a6418c4,50b2c366ec31c06d] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:37:57.591 INFO 13484 --- [ocp-schedules-6,a144c7413a6418c4,50b2c366ec31c06d] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2025-09-02 13:37:57.592 INFO 13484 --- [ForkJoinPool.commonPool-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2025-09-02 13:37:57.592 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2025-09-02 13:37:57.592 INFO 13484 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2025-09-02 13:37:57.592 INFO 13484 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2025-09-02 13:37:57.597 INFO 13484 --- [ForkJoinPool.commonPool-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2025-09-02 13:37:57.597 INFO 13484 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2025-09-02 13:37:57.598 INFO 13484 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2025-09-02 13:37:57.599 INFO 13484 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2025-09-02 13:37:57.599 INFO 13484 --- [ocp-schedules-6,a144c7413a6418c4,50b2c366ec31c06d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2025-09-02 13:37:57.600 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2025-09-02 13:37:57.600 INFO 13484 --- [ForkJoinPool.commonPool-worker-5,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2025-09-02 13:37:57.600 INFO 13484 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2025-09-02 13:37:57.600 INFO 13484 --- [ocp-schedules-6,a144c7413a6418c4,50b2c366ec31c06d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2025-09-02 13:37:57.601 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2025-09-02 13:37:57.623 INFO 13484 --- [ocp-schedules-6,a144c7413a6418c4,50b2c366ec31c06d] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.84:62889/api/v1/log/alarms, params:null 2025-09-02 13:37:57.626 INFO 13484 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.85:62889/api/v1/log/alarms, params:null 2025-09-02 13:37:57.626 INFO 13484 --- [ForkJoinPool.commonPool-worker-5,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.154:62889/api/v1/log/alarms, params:null 2025-09-02 13:37:57.626 INFO 13484 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.19.62.156:62889/api/v1/log/alarms, params:null 2025-09-02 13:37:57.636 INFO 13484 --- [ForkJoinPool.commonPool-worker-7,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://172.17.172.86:62889/api/v1/log/alarms, params:null 2025-09-02 13:37:57.638 INFO 13484 --- [ocp-schedules-6,a144c7413a6418c4,50b2c366ec31c06d] c.o.o.s.common.scheduled.HostSchedules : Get host agent alarms finished. 2025-09-02 13:37:58.280 INFO 13484 --- [ocp-schedules-14,a4ad9978b0a4aebb,d2896e5602baca70] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:37:58.323 INFO 13484 --- [prometheus-minute-schedule-1,152771a22d1d0034,5179a84455e5a92a] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791480, interval=60 2025-09-02 13:37:58.327 INFO 13484 --- [prometheus-minute-schedule-1,152771a22d1d0034,5179a84455e5a92a] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:37:58.434 INFO 13484 --- [ocp-async-10,da5ee1dc15af21f0,985331335e86a582] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:37:58.979 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:37:58.979 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:37:59.095 INFO 13484 --- [ocp-async-1,a64a8278b23255c1,ad74c3ca43b5d1f7] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:37:59.974 INFO 13484 --- [ocp-schedules-22,ba209919c5baf510,dcaa70f89c6a377f] c.o.o.s.common.scheduled.HostSchedules : Check host version schedule, hostIps=[172.19.62.154, 172.19.62.156, 172.17.172.84, 172.17.172.86, 172.17.172.85] 2025-09-02 13:37:59.974 INFO 13484 --- [ocp-schedules-22,ba209919c5baf510,dcaa70f89c6a377f] o.o.c.t.e.u.DistributedExclusiveExecutor : Empty item list, taskName=host-check-agent-version, skip execute 2025-09-02 13:38:00.434 INFO 13484 --- [ocp-async-3,fe0b85ce9509810e,540bc51607d49fa1] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:38:00.834 INFO 13484 --- [ocp-schedules-30,f1a69f2ad7d942c8,53f46905020b9876] c.o.o.s.c.scheduled.TenantSchedules : Collect all tenant major compaction 2025-09-02 13:38:00.834 INFO 13484 --- [ocp-schedules-30,f1a69f2ad7d942c8,53f46905020b9876] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2025-09-02 13:38:00.851 INFO 13484 --- [Collect all tenants major compactions1,f1a69f2ad7d942c8,741cb1ad1ea1f012] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:myoceanbase, tenant:woqu 2025-09-02 13:38:00.852 INFO 13484 --- [Collect all tenants major compactions2,f1a69f2ad7d942c8,313473752c06a1c6] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:myoceanbase, tenant:sys 2025-09-02 13:38:00.871 INFO 13484 --- [Collect all tenants major compactions2,f1a69f2ad7d942c8,313473752c06a1c6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:38:00.872 INFO 13484 --- [Collect all tenants major compactions1,f1a69f2ad7d942c8,741cb1ad1ea1f012] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:38:00.873 INFO 13484 --- [Collect all tenants major compactions2,f1a69f2ad7d942c8,313473752c06a1c6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2025-09-02 13:38:00.873 INFO 13484 --- [Collect all tenants major compactions1,f1a69f2ad7d942c8,741cb1ad1ea1f012] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2025-09-02 13:38:00.877 INFO 13484 --- [Collect all tenants major compactions1,f1a69f2ad7d942c8,741cb1ad1ea1f012] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:myoceanbase, tenant:woqu 2025-09-02 13:38:00.879 INFO 13484 --- [Collect all tenants major compactions2,f1a69f2ad7d942c8,313473752c06a1c6] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:myoceanbase, tenant:sys 2025-09-02 13:38:00.880 INFO 13484 --- [ocp-schedules-30,f1a69f2ad7d942c8,53f46905020b9876] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2025-09-02 13:38:00.889 INFO 13484 --- [ocp-schedules-9,50a23771e59b0554,1909c1a84b19681e] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog instances, size=0 2025-09-02 13:38:00.925 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,f6c7aff43ede3f37,9e7dd7ff917bbbb0] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=f6c7aff43ede3f37, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:38:00.926 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,dde33f9efe87a0b0,6df3b547d63cae50] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=dde33f9efe87a0b0, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:38:00.926 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,fd725482a0a41cfd,2d1db28efb9cc534] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=fd725482a0a41cfd, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:38:00.926 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,e14e9a2e9c638eaf,82984bc6b16f6718] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=e14e9a2e9c638eaf, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:38:00.926 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-3,fd725482a0a41cfd,2d1db28efb9cc534] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=fd725482a0a41cfd, duration=5 ms] 2025-09-02 13:38:00.926 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-5,e14e9a2e9c638eaf,82984bc6b16f6718] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=e14e9a2e9c638eaf, duration=5 ms] 2025-09-02 13:38:00.926 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-7,f6c7aff43ede3f37,9e7dd7ff917bbbb0] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=f6c7aff43ede3f37, duration=5 ms] 2025-09-02 13:38:00.928 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-8,dde33f9efe87a0b0,6df3b547d63cae50] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=dde33f9efe87a0b0, duration=7 ms] 2025-09-02 13:38:01.025 INFO 13484 --- [ocp-schedules-10,2a74ad71dc1cf5b1,04399cd6ab8dc543] c.o.o.s.c.scheduled.ClusterSchedules : Collect all cluster compaction info. 2025-09-02 13:38:01.026 INFO 13484 --- [ocp-schedules-10,2a74ad71dc1cf5b1,04399cd6ab8dc543] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:38:01.027 INFO 13484 --- [Collect compaction for all clusters1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2025-09-02 13:38:01.027 INFO 13484 --- [ocp-schedules-10,2a74ad71dc1cf5b1,04399cd6ab8dc543] .o.o.s.t.b.c.AbstractShardingClusterTask : Collect compaction for all clusters executed completed, allCount=1, successCount=1 2025-09-02 13:38:01.323 INFO 13484 --- [prometheus-minute-schedule-1,ea1bec57e4165dac,864ceadd8b79153d] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791540, interval=60 2025-09-02 13:38:01.328 INFO 13484 --- [prometheus-minute-schedule-1,ea1bec57e4165dac,864ceadd8b79153d] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:38:01.979 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:38:01.980 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:38:02.067 INFO 13484 --- [ocp-schedules-21,c713bf8dad689d86,ed30da32cc914700] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info 2025-09-02 13:38:02.070 INFO 13484 --- [ocp-schedules-21,c713bf8dad689d86,ed30da32cc914700] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Invalidate clusters [] cache info success. 2025-09-02 13:38:02.070 INFO 13484 --- [ocp-schedules-21,c713bf8dad689d86,ed30da32cc914700] c.o.o.c.cache.ConfigServerCacheHolder : [ConfigServerCache]:Refresh all config server cache info success, cached size: 1 2025-09-02 13:38:02.434 INFO 13484 --- [ocp-async-2,c4ca7a03c42fe03b,d56ef65286991c32] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:38:02.797 INFO 13484 --- [ocp-schedules-14,e01f06c423cdd837,e079c930feff0b2a] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:38:02.799 INFO 13484 --- [ocp-schedules-14,e01f06c423cdd837,e079c930feff0b2a] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:38:03.722 INFO 13484 --- [ocp-schedules-21,88baf8dbc5ec9445,4c36813e3522336a] c.o.o.s.c.scheduled.TenantSchedules : Check all tenant service name conflict 2025-09-02 13:38:03.722 INFO 13484 --- [ocp-schedules-21,88baf8dbc5ec9445,4c36813e3522336a] .t.CheckAllTenantServiceNameConflictTask : Check all tenant service name conflict. 2025-09-02 13:38:03.790 INFO 13484 --- [ocp-schedules-6,e466cb89c681673b,9b23ab227052e791] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2025-09-02 13:38:03.798 INFO 13484 --- [ocp-schedules-6,e466cb89c681673b,9b23ab227052e791] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:38:03.798 INFO 13484 --- [ocp-schedules-6,e466cb89c681673b,9b23ab227052e791] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[6bdf2417-5dd5-478e-883b-37f61499e020] 2025-09-02 13:38:04.043 INFO 13484 --- [ocp-schedules-10,fa920e4550f1245d,5a2e2ef996cebcae] c.o.o.s.c.scheduled.TenantSchedules : Check suspend tenant compaction status 2025-09-02 13:38:04.043 INFO 13484 --- [ocp-schedules-10,fa920e4550f1245d,5a2e2ef996cebcae] s.t.b.t.CheckSuspendTenantCompactionTask : Check tenant compaction status. 2025-09-02 13:38:04.081 INFO 13484 --- [ocp-schedules-10,fa920e4550f1245d,5a2e2ef996cebcae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2025-09-02 13:38:04.082 INFO 13484 --- [ocp-schedules-10,fa920e4550f1245d,5a2e2ef996cebcae] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ ocp_query */ tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION 2025-09-02 13:38:04.323 INFO 13484 --- [prometheus-minute-schedule-1,46a103c52776fe5e,84579191286d9910] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791540, interval=60 2025-09-02 13:38:04.328 INFO 13484 --- [prometheus-minute-schedule-1,46a103c52776fe5e,84579191286d9910] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:38:04.434 INFO 13484 --- [ocp-async-8,5f60ff54a215d030,d3ab90e3b0b322b5] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:38:04.936 INFO 13484 --- [ocp-schedules-18,1d34a325467850e2,a1a1cb82ddf3d817] c.o.o.s.c.scheduled.BinlogSchedules : Sync binlog clusters, size=0 2025-09-02 13:38:04.981 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:38:04.981 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:38:05.522 INFO 13484 --- [ocp-schedules-30,f4bfd9833866b304,22f49b4ca6102ce1] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2025-09-02 13:38:05.522 INFO 13484 --- [ocp-schedules-30,f4bfd9833866b304,22f49b4ca6102ce1] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=819357824821161984 2025-09-02 13:38:05.522 INFO 13484 --- [ocp-schedules-30,f4bfd9833866b304,22f49b4ca6102ce1] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2025-09-02 13:38:05.522 INFO 13484 --- [ocp-schedules-30,f4bfd9833866b304,22f49b4ca6102ce1] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=818961159966617600 2025-09-02 13:38:05.523 INFO 13484 --- [ocp-schedules-30,f4bfd9833866b304,22f49b4ca6102ce1] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2025-09-02 13:38:05.523 INFO 13484 --- [ocp-schedules-30,f4bfd9833866b304,22f49b4ca6102ce1] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=818784736907816960 2025-09-02 13:38:06.434 INFO 13484 --- [ocp-async-4,b9970109e535dff4,7153358d6bed3917] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:38:06.985 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:38:06.984936+08:00, lastProcessTime=2025-09-02T13:37:56.984024+08:00 2025-09-02 13:38:07.137 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:38:06.984936+08:00 2025-09-02 13:38:07.283 INFO 13484 --- [ocp-schedules-27,a5b80bb643c8fade,ee3b6530c61a5368] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:38:07.300 WARN 13484 --- [scanner-12211222-1,a5b80bb643c8fade,d7f839b8499a126e] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:38:07.323 INFO 13484 --- [prometheus-minute-schedule-1,ebe81c8821cba201,e0babc5231bc3eb5] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791540, interval=60 2025-09-02 13:38:07.329 INFO 13484 --- [prometheus-minute-schedule-1,ebe81c8821cba201,e0babc5231bc3eb5] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:38:07.982 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:38:07.982 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:38:08.285 INFO 13484 --- [ocp-schedules-7,bb4a19c035c13ac0,2227018a4e357e47] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2025-09-02 13:38:08.435 INFO 13484 --- [ocp-async-6,4e6cf44b6ff326d2,3eb30b2180d6d8ca] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:38:08.637 INFO 13484 --- [ocp-schedules-21,84cc6a042e384597,467e7cfd16c1cfd9] c.o.o.s.c.scheduled.ClusterSchedules : Sync root server list. 2025-09-02 13:38:08.639 INFO 13484 --- [ocp-schedules-21,84cc6a042e384597,467e7cfd16c1cfd9] .o.o.s.t.b.c.AbstractShardingClusterTask : Consume clusters: [1] 2025-09-02 13:38:08.652 INFO 13484 --- [ocp-schedules-21,84cc6a042e384597,467e7cfd16c1cfd9] .o.o.s.t.b.c.AbstractShardingClusterTask : Refresh root service list executed completed, allCount=1, successCount=1 2025-09-02 13:38:09.095 INFO 13484 --- [ocp-async-9,7939824d7e4788ee,b34fb22151d1e309] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2025-09-02 13:38:09.267 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule start 2025-09-02 13:38:09.280 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=1, name=myoceanbase, obClusterId=1755130302, status=RUNNING} 2025-09-02 13:38:09.292 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (myoceanbase:1755130302-1) 2025-09-02 13:38:09.292 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (myoceanbase:1755130302-2) 2025-09-02 13:38:09.293 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (myoceanbase:1755130302-3) 2025-09-02 13:38:09.294 INFO 13484 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=3, level=0->1, c=0) - not enough cached interval (source=2025-09-02T13:35:30, target=2025-09-02T13:33:30, granule=120000ms, delay=120000ms) 2025-09-02 13:38:09.294 INFO 13484 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=3, level=1->2, c=0) - not enough cached interval (source=2025-09-02T13:33:30, target=2025-09-02T13:28:00, granule=600000ms, delay=330000ms) 2025-09-02 13:38:09.294 INFO 13484 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=1, level=0->1, c=0) - not enough cached interval (source=2025-09-02T13:35:30, target=2025-09-02T13:33:30, granule=120000ms, delay=120000ms) 2025-09-02 13:38:09.295 INFO 13484 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=1, level=1->2, c=0) - not enough cached interval (source=2025-09-02T13:33:30, target=2025-09-02T13:27:00, granule=600000ms, delay=390000ms) 2025-09-02 13:38:09.295 INFO 13484 --- [sql-rollup-worker-0,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=2, level=0->1, c=0) - not enough cached interval (source=2025-09-02T13:35:30, target=2025-09-02T13:33:30, granule=120000ms, delay=120000ms) 2025-09-02 13:38:09.295 INFO 13484 --- [sql-rollup-worker-0,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=myoceanbase:1755130302, server=2, level=1->2, c=0) - not enough cached interval (source=2025-09-02T13:33:30, target=2025-09-02T13:27:30, granule=600000ms, delay=360000ms) 2025-09-02 13:38:09.295 INFO 13484 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule end, elapsed=28ms, period=120000ms 2025-09-02 13:38:10.004 INFO 13484 --- [ocp-schedules-2,,] c.o.o.s.c.scheduled.ObproxySchedules : Check obproxy server stats, size=2 2025-09-02 13:38:10.005 INFO 13484 --- [ocp-schedules-29,,] c.o.o.s.c.scheduled.ObproxySchedules : Collect obproxy metrics, size=2 2025-09-02 13:38:10.019 INFO 13484 --- [Check obproxy1,e281da7951baead5,7bd7ec41ae3b0986] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:38:10.019 INFO 13484 --- [Check obproxy1,e281da7951baead5,7bd7ec41ae3b0986] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:38:10.021 INFO 13484 --- [Check obproxy2,7ef7ac2cd055b7ab,38547a61972121d4] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:38:10.021 INFO 13484 --- [Check obproxy2,7ef7ac2cd055b7ab,38547a61972121d4] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:38:10.024 INFO 13484 --- [Collect obproxy2,fef1e6c1e0478cdf,3b47f97739aa5708] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.156 2883 2025-09-02 13:38:10.024 INFO 13484 --- [Collect obproxy1,30ec3df2e8450fe6,df39db463a09c4f0] c.o.o.s.o.SystemOdpOperatorFactory : connect obproxy 172.19.62.154 2883 2025-09-02 13:38:10.024 INFO 13484 --- [Collect obproxy2,fef1e6c1e0478cdf,3b47f97739aa5708] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:38:10.024 INFO 13484 --- [Collect obproxy1,30ec3df2e8450fe6,df39db463a09c4f0] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig, args: [] 2025-09-02 13:38:10.045 INFO 13484 --- [Check obproxy1,e281da7951baead5,7bd7ec41ae3b0986] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:38:10.047 INFO 13484 --- [Check obproxy2,7ef7ac2cd055b7ab,38547a61972121d4] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [proxy_id] 2025-09-02 13:38:10.048 INFO 13484 --- [Collect obproxy1,30ec3df2e8450fe6,df39db463a09c4f0] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:38:10.048 INFO 13484 --- [Check obproxy1,e281da7951baead5,7bd7ec41ae3b0986] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 1 success, update last active time to now 2025-09-02 13:38:10.049 INFO 13484 --- [Check obproxy1,e281da7951baead5,7bd7ec41ae3b0986] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 1 proxy_id: 6, need to sync to meta: false 2025-09-02 13:38:10.050 INFO 13484 --- [Collect obproxy2,fef1e6c1e0478cdf,3b47f97739aa5708] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%proxy_mem_limited%] 2025-09-02 13:38:10.051 INFO 13484 --- [Check obproxy2,7ef7ac2cd055b7ab,38547a61972121d4] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 2 success, update last active time to now 2025-09-02 13:38:10.051 INFO 13484 --- [Check obproxy2,7ef7ac2cd055b7ab,38547a61972121d4] c.o.o.o.internal.ObproxyServiceImpl : Obproxy 2 proxy_id: 5, need to sync to meta: false 2025-09-02 13:38:10.052 INFO 13484 --- [Collect obproxy1,30ec3df2e8450fe6,df39db463a09c4f0] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:38:10.053 INFO 13484 --- [Collect obproxy2,fef1e6c1e0478cdf,3b47f97739aa5708] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%max_connections%] 2025-09-02 13:38:10.055 INFO 13484 --- [Collect obproxy1,30ec3df2e8450fe6,df39db463a09c4f0] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:38:10.057 INFO 13484 --- [Collect obproxy2,fef1e6c1e0478cdf,3b47f97739aa5708] com.oceanbase.ocp.odpsdk.OdpOperator : [odpsdk] sql: SHOW proxyconfig LIKE ?, args: [%client_max_connections%] 2025-09-02 13:38:10.323 INFO 13484 --- [prometheus-minute-schedule-1,69cc20349019bd6a,d355fb38453affca] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791540, interval=60 2025-09-02 13:38:10.327 INFO 13484 --- [prometheus-minute-schedule-1,69cc20349019bd6a,d355fb38453affca] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:38:10.435 INFO 13484 --- [ocp-async-8,1858185317464003,a532c89704ead7eb] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:38:10.553 INFO 13484 --- [async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 11, ruleSize = 11. 2025-09-02 13:38:10.921 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,de1080031d23c276,ab67c52fb854ac47] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=cluster%2Ctenant%2Cobproxy%2Chost%2Calarm%2Ctask, client=114.221.202.171, traceId=de1080031d23c276, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[cluster, tenant, obproxy, host, alarm, task],] 2025-09-02 13:38:10.921 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,b0ccad84bd26f1be,76631d613b0da5ae] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=backup, client=114.221.202.171, traceId=b0ccad84bd26f1be, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[backup],] 2025-09-02 13:38:10.921 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,65d79078522a517d,62557ca6f436679d] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/status?modules=performance, client=114.221.202.171, traceId=65d79078522a517d, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.getOverviewStatus(List), args=[performance],] 2025-09-02 13:38:10.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-4,b0ccad84bd26f1be,76631d613b0da5ae] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=b0ccad84bd26f1be, duration=4 ms] 2025-09-02 13:38:10.922 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-6,65d79078522a517d,62557ca6f436679d] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=65d79078522a517d, duration=4 ms] 2025-09-02 13:38:10.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-1,de1080031d23c276,ab67c52fb854ac47] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/status client=114.221.202.171, traceId=de1080031d23c276, duration=6 ms] 2025-09-02 13:38:10.924 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,765ba6f2dff1ea6e,5718266dd059ef91] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /api/v2/overview/resourceStats?modules=cluster, client=114.221.202.171, traceId=765ba6f2dff1ea6e, method=SuccessResponse com.oceanbase.ocp.server.common.controller.OcpOverviewController.resourceStatistic(List), args=[cluster],] 2025-09-02 13:38:10.932 INFO 13484 --- [http-nio-0.0.0.0-8080-exec-2,765ba6f2dff1ea6e,5718266dd059ef91] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /api/v2/overview/resourceStats client=114.221.202.171, traceId=765ba6f2dff1ea6e, duration=14 ms] 2025-09-02 13:38:10.982 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:38:10.984 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:38:11.969 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.o.s.common.scheduled.CacheSchedules : Refresh cache bean: hostQueryCache 2025-09-02 13:38:11.969 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.o.core.cache.helper.HostQueryCache : Init host cache. 2025-09-02 13:38:11.975 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.o.core.cache.helper.HostQueryCache : Finished init host cache. 2025-09-02 13:38:11.975 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.o.s.common.scheduled.CacheSchedules : Refresh cache bean: binlogQueryCache 2025-09-02 13:38:11.975 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.o.c.cache.helper.BinlogQueryCache : Init binlog cache. 2025-09-02 13:38:11.983 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.o.c.cache.helper.BinlogQueryCache : Finish init binlog cache. 2025-09-02 13:38:11.983 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.o.s.common.scheduled.CacheSchedules : Refresh cache bean: obQueryCache 2025-09-02 13:38:11.983 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.ocp.core.cache.helper.ObQueryCache : Init ob cache. 2025-09-02 13:38:11.995 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.ocp.core.cache.helper.ObQueryCache : Finished init ob cache. 2025-09-02 13:38:11.995 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.o.s.common.scheduled.CacheSchedules : Refresh cache bean: obProxyQueryCache 2025-09-02 13:38:11.995 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.o.c.cache.helper.ObProxyQueryCache : Init obproxy cache. 2025-09-02 13:38:12.002 INFO 13484 --- [ocp-schedules-24,13d7a3c326db304c,d8a1983d2fd865c5] c.o.o.c.cache.helper.ObProxyQueryCache : Finished init obproxy cache. 2025-09-02 13:38:12.435 INFO 13484 --- [ocp-async-4,1a9379ecc7d54441,167c4d84fcb38cbd] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:38:12.805 INFO 13484 --- [ocp-schedules-19,67a73f90fb09504f,90b49d052da8101b] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration service group 2025-09-02 13:38:12.807 INFO 13484 --- [ocp-schedules-19,67a73f90fb09504f,90b49d052da8101b] o.i.a.t.CheckArbitrationServiceGroupTask : No ob server lose connection with its arbitration 2025-09-02 13:38:13.323 INFO 13484 --- [prometheus-minute-schedule-1,753251635c4b8579,428026b57390bf91] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791540, interval=60 2025-09-02 13:38:13.328 INFO 13484 --- [prometheus-minute-schedule-1,753251635c4b8579,428026b57390bf91] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:38:13.535 INFO 13484 --- [metric-archive-second-8,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1756791420, nodeCount=678, affectRows=678 2025-09-02 13:38:13.538 INFO 13484 --- [metric-archive-second-8,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1756791420, nodeCount=676, validCount=674 2025-09-02 13:38:13.540 INFO 13484 --- [metric-archive-second-3,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1756791420, nodeCount=1000, affectRows=1000 2025-09-02 13:38:13.542 INFO 13484 --- [metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1756791420, nodeCount=1000, affectRows=1000 2025-09-02 13:38:13.546 INFO 13484 --- [metric-archive-second-3,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1756791420, nodeCount=997, validCount=996 2025-09-02 13:38:13.547 INFO 13484 --- [metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1756791420, nodeCount=997, validCount=994 2025-09-02 13:38:13.683 INFO 13484 --- [ocp-schedules-8,416129f1310252bf,5e826b51c647a07b] c.o.o.alarm.service.OcpAlarmServiceImpl : fix alarm target, alarms size 8 2025-09-02 13:38:13.689 INFO 13484 --- [ocp-schedules-29,ab154f3f74b42bb4,79b676f1f7e2e033] c.o.o.s.c.scheduled.ClusterSchedules : Check arbitration auto replace status 2025-09-02 13:38:13.691 INFO 13484 --- [ocp-schedules-16,04aedcf911041370,ec5c44cf8303231e] c.o.o.s.c.scheduled.ClusterSchedules : Check certificate expired. 2025-09-02 13:38:13.984 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:38:13.984 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:38:14.435 INFO 13484 --- [ocp-async-2,2e8f2d21e8929450,c9909de00160cc75] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:38:15.795 INFO 13484 --- [metric-archive-async-6,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1756791420, nodeCount=383, affectRows=383 2025-09-02 13:38:16.323 INFO 13484 --- [prometheus-minute-schedule-1,46118801c32697c3,e8b5a045178a966d] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1756791540, interval=60 2025-09-02 13:38:16.327 INFO 13484 --- [prometheus-minute-schedule-1,46118801c32697c3,e8b5a045178a966d] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=13, collectTaskCount=13 2025-09-02 13:38:16.435 INFO 13484 --- [ocp-async-10,5ca1b8d4184e72e4,47f30c32f9a7c349] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2025-09-02 13:38:16.985 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 172.19.62.162:8080 refreshes policies... 2025-09-02 13:38:16.985 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2025-09-02T13:38:16.985068+08:00, lastProcessTime=2025-09-02T13:38:06.984936+08:00 2025-09-02 13:38:16.985 INFO 13484 --- [pool-11-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2025-09-02 13:38:17.136 INFO 13484 --- [alarm-task-1,398e271d7576b4c9,8509dde6f7d8e2df] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2025-09-02T13:38:16.985068+08:00 2025-09-02 13:38:17.360 INFO 13484 --- [ocp-schedules-18,a97c3263885f8a40,b338909f070e88c2] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2025-09-02 13:38:17.395 WARN 13484 --- [scanner-12212034-1,a97c3263885f8a40,b0f5b0fe5399738a] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 14 2025-09-02 13:38:17.396 INFO 13484 --- [alarm-detect-19,a97c3263885f8a40,147223bc7a7bdb60] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 3016188, data is empty. 2025-09-02 13:38:17.429 INFO 13484 --- [alarm-detect-2,a97c3263885f8a40,a09829692ebdbe26] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 3016188, data is empty. 2025-09-02 13:38:17.429 INFO 13484 --- [alarm-detect-18,a97c3263885f8a40,93641d46f597a4c8] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 3016188, data is empty. 2025-09-02 13:38:17.515 INFO 13484 --- [alarm-detect-9,a97c3263885f8a40,8e68e4de6f030340] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 3016188, data is empty. 2025-09-02 13:38:18.117 INFO 13484 --- [ocp-schedules-4,2166bd5fc7db7b85,c89b85f702dc5a64] c.o.o.s.common.scheduled.PerfSchedules : Oas sql curing, clusters=[]