2024-03-13 15:54:41.887 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804482,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:43.108 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:43.113 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:43.655 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,6d436924921a4747,6c9b1bd2cacf] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.142, traceId=6d436924921a4747, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:54:43.660 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,6d436924921a4747,6c9b1bd2cacf] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.142, traceId=6d436924921a4747, duration=6 ms] 2024-03-13 15:54:43.887 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804505,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:45.888 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804528,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:46.102 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804532,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:54:46.113 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:46.114 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:46.748 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:54:46.747757+08:00, lastProcessTime=2024-03-13T15:54:36.746676+08:00 2024-03-13 15:54:46.864 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:54:46.747757+08:00 2024-03-13 15:54:46.910 INFO 1433467 --- [pool-ocp-schedules-13,9f1fa9f82daf4bae,cdaa10dd1e44] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:54:47.888 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804552,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:47.908 INFO 1433467 --- [pool-clean-time-series-data15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:54:47.924 INFO 1433467 --- [pool-print-time-series-interval15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316487: {1710316380=1394, 1710316395=1394, 1710316410=1394, 1710316425=1394, 1710316440=1394, 1710316455=1394, 1710316470=1394, 1710316485=60}, tsDataMapSize: 1122696, readyDataCacheSize: 8288 2024-03-13 15:54:47.948 INFO 1433467 --- [pool-clean-time-series-data60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:54:47.949 INFO 1433467 --- [pool-print-time-series-interval60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316487: {1710316320=32, 1710316380=34, 1710316440=6}, tsDataMapSize: 17768, readyDataCacheSize: 18488 2024-03-13 15:54:49.114 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:49.115 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:49.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804569,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:54:49.321 WARN 1433467 --- [pool-scanner-1609791898-1,e615076dce214056,570005faba11] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:54:49.412 INFO 1433467 --- [alarm-detect-17,f1a954638fdb409a,0eb0dc68bffc] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000911, data is empty. 2024-03-13 15:54:49.415 INFO 1433467 --- [alarm-detect-17,f1a954638fdb409a,0eb0dc68bffc] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000911, data is empty. 2024-03-13 15:54:49.456 INFO 1433467 --- [alarm-detect-11,aaaa33118a1e4658,7de15b276b07] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316455 ~ 1710316479 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316470=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:54:49.456 INFO 1433467 --- [alarm-detect-11,aaaa33118a1e4658,7de15b276b07] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316455 ~ 1710316479 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316470=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:54:49.888 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804576,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:51.889 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804599,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:52.116 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:52.117 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:52.719 INFO 1433467 --- [pool-ocp-schedules-3,d61bd653daa749fc,bdc27671a07c] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:54:52.726 INFO 1433467 --- [pool-ocp-schedules-3,d61bd653daa749fc,bdc27671a07c] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:54:52.726 INFO 1433467 --- [pool-ocp-schedules-3,d61bd653daa749fc,bdc27671a07c] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:54:53.889 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804622,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:55.117 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:55.118 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:55.688 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2024-03-13 15:54:55.688 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=609459010050985984 2024-03-13 15:54:55.699 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2024-03-13 15:54:55.699 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624101396131614721 2024-03-13 15:54:55.707 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2024-03-13 15:54:55.707 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624425259050733570 2024-03-13 15:54:55.889 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804645,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:56.103 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804649,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:54:56.125 INFO 1433467 --- [pool-Check obproxy13,18b3329f433b446c,9509184b4a96] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:54:56.127 INFO 1433467 --- [pool-Check obproxy14,18b3329f433b446c,0aa0a18a242f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:54:56.127 INFO 1433467 --- [pool-Check obproxy8,18b3329f433b446c,6a847e76f048] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:54:56.128 INFO 1433467 --- [pool-Check obproxy12,18b3329f433b446c,3d7db83b0c7f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:54:56.746 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:54:56.745596+08:00, lastProcessTime=2024-03-13T15:54:46.747757+08:00 2024-03-13 15:54:56.828 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:54:56.745596+08:00 2024-03-13 15:54:56.916 INFO 1433467 --- [pool-ocp-schedules-13,9a736d98159a4fc2,172b642b20e7] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:54:57.890 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804668,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:58.118 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:58.119 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:59.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804685,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:54:59.281 INFO 1433467 --- [alarm-detect-11,cdc71058cc8e4188,6e4abc818365] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000911, data is empty. 2024-03-13 15:54:59.282 INFO 1433467 --- [alarm-detect-17,9afb90b55ab14982,14a686aeecf6] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000911, data is empty. 2024-03-13 15:54:59.283 INFO 1433467 --- [alarm-detect-25,39bab5caafb84041,f17cb15cd0cb] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000911, data is empty. 2024-03-13 15:54:59.286 INFO 1433467 --- [alarm-detect-11,cdc71058cc8e4188,6e4abc818365] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000911, data is empty. 2024-03-13 15:54:59.287 INFO 1433467 --- [alarm-detect-17,9afb90b55ab14982,14a686aeecf6] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000911, data is empty. 2024-03-13 15:54:59.287 INFO 1433467 --- [alarm-detect-25,39bab5caafb84041,f17cb15cd0cb] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000911, data is empty. 2024-03-13 15:54:59.329 WARN 1433467 --- [pool-scanner-1609792550-1,1c87fe138a164bf1,5eabab7754e7] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:54:59.442 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316470 ~ 1710316489 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316485=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:54:59.442 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316470 ~ 1710316489 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316485=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:54:59.442 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316470 ~ 1710316489 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316485=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:54:59.442 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316470 ~ 1710316489 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316485=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:54:59.442 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316470 ~ 1710316489 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316485=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:54:59.443 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.a.core.detect.AlarmRuleDetector : detectByMetric done, alarmType=host_ntp_service_not_exist, needSendingCount=1, detectAlarmsCount=1, matchedTargets=[alarm_template_id=3:host=10.10.10.148-2, alarm_template_id=3:host=10.10.10.147-2, alarm_template_id=3:host=10.10.10.146-2, alarm_template_id=3:host=10.10.10.145-2, alarm_template_id=3:host=10.10.10.144-2] 2024-03-13 15:54:59.891 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804692,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:00.497 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:00.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.497 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=38, name=Detect alarms on schedule, templateName=BackupScheduleAlarmTemplateFactory, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:54:00.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.500 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Detect alarms, nodeCount=1, taskCount=1 2024-03-13 15:55:00.501 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.501 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Detect alarms, taskDefinitionId=38, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.508 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:50:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.509 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999977, name=DiagFuncUpdateSchedule, templateName=DiagFuncUpdateSchedule, scheduleRule=0 */5 * * * *, lastRunTime=2024-03-13 15:50:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.510 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=DiagFuncUpdateSchedule, nodeCount=1, taskCount=1 2024-03-13 15:55:00.510 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.510 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=DiagFuncUpdateSchedule, taskDefinitionId=999977, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.514 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:50:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.514 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999978, name=HighRiskSqlSchedule, templateName=HighRiskSqlSchedule, scheduleRule=0 */5 * * * *, lastRunTime=2024-03-13 15:50:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.539 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=HighRiskSqlAuditStatTaskSchedule, nodeCount=1, taskCount=1 2024-03-13 15:55:00.539 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.539 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=HighRiskSqlAuditStatTaskSchedule, taskDefinitionId=999978, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.543 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:50:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.543 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999980, name=Collect all tenants mini compactions, templateName=CollectTenantMiniCompactionSchedule, scheduleRule=0 */5 * * * *, lastRunTime=2024-03-13 15:50:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.545 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all tenants mini compactions, nodeCount=1, taskCount=1 2024-03-13 15:55:00.545 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.545 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all tenants mini compactions, taskDefinitionId=999980, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.549 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:00.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.549 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999981, name=Collect all tenants major compactions, templateName=CollectTenantCompactionSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:54:00.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.550 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all tenants major compactions, nodeCount=1, taskCount=1 2024-03-13 15:55:00.550 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.550 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all tenants major compactions, taskDefinitionId=999981, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.554 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:00.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.554 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999982, name=Collect SlowSql to SqlAggregateData for alarm, templateName=CollectSlowSqlSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:54:00.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.559 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect SlowSql to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:55:00.559 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.559 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect SlowSql to SqlAggregateData for alarm, taskDefinitionId=999982, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.564 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.564 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999983, name=Collect trans stat to SqlAggregateData for alarm, templateName=CollectTransSatatSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:54:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.566 INFO 1433467 --- [pool-prometheus-minute-schedule-1,fbce573c7eb84c5f,f6cfe8b80370] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1710316560, interval=60 2024-03-13 15:55:00.567 INFO 1433467 --- [pool-prometheus-cache-monitor1,04c2f32a4ca04484,262ae173d066] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":12072,"minuteWriteQueue":0,"minuteReadCacheSize":12328,"secondWriteCacheSize":3332,"seriesIdKeySize":67930,"secondWriteQueue":0,"minuteWriteCacheSize":12393} 2024-03-13 15:55:00.568 INFO 1433467 --- [pool-prometheus-cache-monitor1,04c2f32a4ca04484,262ae173d066] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{go_memory_classes_heap_stacks_bytes=15, ob_sysstat_40012=19, ob_server_resource_memory_assigned_bytes=13, ob_sysstat_40011=19, ob_sysstat_40010=19, ob_unit_num=13, ob_database_disk_required_size=35, go_memstats_mspan_sys_bytes=15, go_memstats_stack_sys_bytes=15, ob_tenant_request_queue_total_size=18, jvm_gc_memory_allocated_bytes_total=3, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=18, ob_sysstat_40019=19, observer_fd_count=13, ob_sysstat_40018=19, processor_process_metrics_total=21, jvm_gc_max_data_size_bytes=3, go_memstats_frees_total=15, sql_audit_input_collect_data_jumped_total=8, host_available=64, mysql_output_write_sql_failed_count=17, ob_database_disk_data_size=34, ob_system_event_detail_total_timeouts=763, ob_tenant_memstore_freeze_cnt=14, ob_latch_sleep_total=312, ob_tenant_unit_min_memory=25, ob_sysstat_60087=19, ob_memstore_total_bytes=20, metric_cardinality_count=12, process_resident_memory_bytes=15, jvm_threads_states_threads=8, ob_cache_size_bytes=34, ob_tenant_unit_max_memory=25, input_collect_metrics_total=26, ob_sysstat_60083=19, ob_sysstat_50039=19, ob_sysstat_50038=19, node_disk_io_time_weighted_seconds_total=33, process_open_fds=15, ob_sysstat_50037=19, process_memory_rss_bytes=33, ob_server_resource_cpu_assigned_percent=13, ob_sysstat_50030=19, go_memory_classes_profiling_buckets_bytes=15, go_memstats_heap_inuse_bytes=15, ob_sysstat_40118=19, ob_sysstat_40117=19, ob_sysstat_40116=19, node_memory_memfree_bytes=26, ob_sysstat_40005=19, process_exists=43, ob_sysstat_40004=19, ob_sysstat_40003=19, ob_sysstat_40002=19, ob_sysstat_40001=19, go_gc_duration_seconds=20, go_memory_classes_metadata_mspan_inuse_bytes=15, ob_server_resource_disk_bytes=13, ob_sysstat_40000=19, odp_current_session=26, process_thread_count=33, ob_sysstat_50040=19, ob_sysstat_40009=19, ob_sysstat_40008=19, ob_disk_free_bytes=13, node_filesystem_avail_bytes=72, ob_sysstat_40007=19, ob_sysstat_40006=19, ob_backup_storage_capacity_bytes=650, ob_unit_config_max_iops=21, ob_plan_cache_memory_bytes=18, node_network_transmit_errs_total=28, go_gc_duration_seconds_count=15, monagent_oceanbase_plan_explain_size=4, ob_sysstat_50011=19, ob_sysstat_50010=19, node_disk_read_bytes_total=33, ob_compaction_is_suspended=19, mysql_output_write_sql_milliseconds_summary_count=17, go_memstats_lookups_total=15, mysql_output_write_sql_count=17, process_cpu_percent=33, ob_sysstat_60057=19, host_agent_process_status=18, ob_server_resource_cpu=13, ob_tenant_server_data_index_row_count=20, go_memstats_alloc_bytes=15, go_sched_gomaxprocs_threads=15, node_cpu_seconds_total_native=38, ob_sysstat_140002=19, ob_sysstat_140005=19, ob_sysstat_140006=19, ob_sysstat_140003=19, ob_tenant_resource_min_memory=18, ob_memstore_freeze_times=20, ob_sysstat_50022=19, jvm_gc_pause_seconds_count=8, ob_sysstat_50021=19, jvm_gc_overhead_percent=3, go_memory_classes_heap_unused_bytes=15, ob_cluster_status_check=13, jvm_memory_committed_bytes=10, node_net_bandwidth_bps=29, monagent_oceanbase_db_pool_idle=4, ob_tenant_server_data_index_data_size=20, go_gc_heap_allocs_by_size_bytes_total_count=15, go_memstats_heap_idle_bytes=15, go_gc_cycles_forced_gc_cycles_total=15, go_gc_pauses_seconds_total_bucket=26, ob_server_resource_memory_bytes=13, ob_tenant_assigned_mem_assigned=18, ob_tenant_memstore_minor_freeze_times=14, ob_tenant_server_required_size=22, ob_sysstat_60031=19, process_fd_count=33, ob_tenant_unit_max_cpu=25, ob_latch_wait_seconds_total=312, go_memory_classes_metadata_mcache_free_bytes=15, node_memory_buffers_bytes=26, ob_sysstat_50009=19, go_memstats_heap_released_bytes=15, node_network_receive_bytes_total=28, ob_sysstat_50005=19, ob_sysstat_50008=19, ob_sysstat_50001=19, ob_sysstat_50004=19, exposer_request_latencies_count=6, ob_sysstat_50000=19, ob_session_active_num=18, mysql_output_context_timeout_discard_metrics_count=9, go_memory_classes_metadata_mcache_inuse_bytes=15, node_disk_read_time_seconds_total=33, ob_sysstat_60021=19, ob_sysstat_60020=19, ob_sysstat_60024=19, ob_compaction_frozen_version=19, ob_sysstat_60023=19, metric_count=15, ob_sysstat_60022=19, obproxysys_connectable=1556, ob_tenant_assigned_cpu_assigned=18, mysql_output_write_sql_retry_count=17, ob_tenant_unit_min_cpu=25, ob_unit_config_max_cpu=21, http_request_milliseconds_summary_count=75, monagent_oceanbase_audit_timeinfo_cache_size=4, http_request_milliseconds_summary=78, ob_latch_immediate_miss_total=312, ob_latch_spin_get_total=312, ob_log_level_count=17, monagent_pipeline_report_metrics_total=16, exposer_transferred_bytes_total=6, ob_tenant_context_memory_count=841, node_disk_reads_completed_total=33, ob_sysstat_60019=19, ob_sysstat_60003=19, ob_sysstat_60002=19, ob_sysstat_60001=19, ob_sysstat_60000=19, ob_latch_miss_total=312, go_sched_latencies_seconds_count=15, ocp_meta_db_health=5, ob_disk_total_bytes=13, go_memstats_heap_objects=15, go_memstats_mcache_inuse_bytes=15, ob_sysstat_60005=19, ob_sysstat_60004=19, go_memory_classes_other_bytes=15, ob_memstore_active_bytes=20, go_memstats_alloc_bytes_total=15, ob_tenant_server_partition_replica_count=21, process_count=34, ob_system_event_detail_total_waits=763, odp_used_connections=25, backup_storage_capacity_used_percent=8, go_gc_heap_frees_objects_total=15, go_memory_classes_metadata_mspan_free_bytes=15, ob_disk_log_used_bytes=13, ob_compaction_is_error=19, go_gc_limiter_last_enabled_gc_cycle=15, monagent_pipeline_execute_total=24, node_filesystem_files=86, process_start_time_seconds=15, go_memstats_heap_alloc_bytes=15, ob_tenant_resource_max_cpu=18, ntp_service_available=26, go_memstats_mspan_inuse_bytes=15, mysql_output_write_sql_milliseconds_summary=20, go_gc_heap_frees_by_size_bytes_total_count=15, monagent_pipeline_execute_seconds_total=24, ob_sysstat_80001=19, go_memstats_sys_bytes=15, go_memory_classes_heap_free_bytes=15, system_load_average_1m=3, monagent_oceanbase_audit_stat_cache_size=4, ob_tenant_memory_used_bytes=22, obproxy_server_max_connections=7, ob_sysstat_80002=19, go_memstats_gc_cpu_fraction=15, ob_sysstat_80003=19, monagent_oceanbase_audit_cache_size=4, ob_tenant_context_memory_used_bytes=841, ob_sysstat_80007=19, node_disk_writes_completed_total=33, ob_sysstat_80008=19, ob_sysstat_80009=19, go_sched_goroutines_goroutines=15, go_memory_classes_metadata_other_bytes=15, go_gc_cycles_total_gc_cycles_total=15, system_cpu_usage=3, process_cpu_usage=3, node_disk_written_bytes_total=33, monagent_oceanbase_ash_cache_size=4, go_goroutines=15, ob_sysstat_80023=19, exposer_request_latencies_sum=6, ob_unit_config_min_cpu=21, process_virtual_memory_max_bytes=15, go_gc_heap_frees_bytes_total=15, ob_tenant500_memory_hold_bytes=13, exposer_scrapes_total=6, go_gc_heap_allocs_by_size_bytes_total_bucket=27, ob_sysstat_80025=19, ob_sysstat_80026=19, oceanbase_connectivity=15, node_memory_sreclaimable_bytes=26, go_sched_latencies_seconds_sum=15, tcp_retrans=25, exporter_export_metrics_bytes_total=26, node_filesystem_size_bytes=72, http_server_requests_seconds_count=203, ob_compaction_frozen_time=19, monagent_input_sql_duration_seconds_sum=55, log_stream_degraded_count=18, ob_plan_cache_hit_total=18, node_filesystem_readonly=72, ob_sysstat_80040=19, ob_sysstat_80041=19, io_await=52, go_gc_heap_allocs_objects_total=15, go_memstats_mallocs_total=15, ob_unit_config_max_memory_bytes=21, go_gc_stack_starting_size_bytes=15, io_util=52, ob_sysstat_200002=19, ob_sysstat_200001=19, ob_tenant_disk_v4_data_size=18, go_sched_latencies_seconds_bucket=26, ob_tenant_assigned_cpu_total=18, ob_tenant_server_available_disk_free=19, node_disk_write_time_seconds_total=33, cpu_count=25, output_write_metrics_total=9, ob_cluster_sync=15, go_gc_heap_objects_objects=15, odp_transaction_total=16, ob_system_event_detail_time_waited_us=763, process_max_fds=15, http_status_count=20, ob_sysstat_80057=19, ocp_alarm_detect_duration_ms_count=115, go_gc_heap_allocs_by_size_bytes_total_sum=15, go_memstats_mcache_sys_bytes=15, ob_tenant_disk_v4_log_size=18, go_cgo_go_to_c_calls_calls_total=15, ocp_alarm_detect_duration_ms_sum=115, jvm_threads_live_threads=3, node_memory_cached_bytes=26, ob_table_num=16, ob_sysstat_10000=19, ob_sysstat_10006=19, ob_sysstat_10005=19, ob_sysstat_10002=19, ob_sysstat_10001=19, go_memstats_last_gc_time_seconds=15, ob_sysstat_10004=19, ob_sysstat_10003=19, monitor_exporter_avaliable=62, log_tailer_count=15, ob_server_num=25, ob_session_all_num=18, ob_system_event_time_waited=24, ob_tenant_resource_min_cpu=18, monagent_plugin_execute_seconds_total=39, go_gc_cycles_automatic_gc_cycles_total=15, go_gc_heap_allocs_bytes_total=15, go_memstats_gc_sys_bytes=15, ob_latch_get_total=312, go_info=16, log_tailer_processed_file_count=21, node_load15=24, ob_disk_log_assigned_bytes=13, go_memory_classes_heap_released_bytes=15, jvm_memory_max_bytes=10, ob_unit_config_min_iops=21, ob_sysstat_30080=19, ob_sysstat_30082=19, ob_sysstat_30081=19, ob_sysstat_30084=19, ob_sysstat_30083=19, ob_sysstat_30086=19, ob_sysstat_30085=19, ob_tenant_log_disk_total_bytes=18, ob_memstore_freeze_trigger_bytes=20, ob_waitevent_wait_seconds_total=18, ob_server_stopped_duration_seconds=15, go_memstats_next_gc_bytes=15, node_filesystem_files_free=86, ob_memtable_max_snapshot_duration_seconds=18, monagent_oceanbase_audit_unique_key_cache_size=4, process_virtual_memory_bytes=15, ob_sysstat_20002=19, net_connectivity=15, ob_sysstat_20001=19, monagent_input_sql_duration_seconds_bucket=65, ob_latch_immediate_get_total=312, node_cpu_seconds_total=32, ob_sysstat_20000=19, ob_index_error_num=13, ob_tenant_server_available_disk_total=19, ob_tenant_memory_hold_bytes=22, ob_sysstat_30066=19, go_memstats_buck_hash_sys_bytes=15, mysql_output_write_sql_milliseconds_summary_sum=17, ob_tenant_disk_total_size=18, odp_sql_request_total=53, mysql_output_write_task_size=17, exporter_export_metrics_total=26, ob_disk_log_total_bytes=13, system_cpu_count=3, ob_sysstat_130004=19, process_uptime_seconds=33, node_ntp_offset_seconds=25, go_memstats_heap_sys_bytes=15, ob_sysstat_140012=19, ob_sysstat_140013=19, jvm_gc_pause_seconds_max=8, ob_tenant_request_queue_large_size=18, ob_sysstat_130000=19, go_memstats_other_sys_bytes=15, ob_sysstat_130001=19, jvm_memory_used_bytes=10, ob_sysstat_130002=19, go_threads=15, http_request_milliseconds_summary_sum=75, http_server_requests_seconds_sum=203, ob_tenant_assigned_mem_total=18, node_memory_memtotal_bytes=26, node_network_transmit_drop_total=28, go_gc_pauses_seconds_total_count=15, exposer_request_latencies=9, ob_waitevent_wait_total=18, go_gc_heap_goal_bytes=15, ob_tenant_log_disk_used_bytes=18, jvm_gc_memory_promoted_bytes_total=3, odp_sql_cost_total=51, ob_sysstat_30011=19, ob_sysstat_30010=19, ob_sysstat_30013=19, ob_sysstat_30012=19, ob_tenant500_memory_used_bytes=13, node_network_transmit_bytes_total=28, node_network_receive_drop_total=28, go_gc_heap_tiny_allocs_objects_total=15, go_gc_duration_seconds_sum=15, ob_server_resource_memory_assigned_percent=13, jvm_gc_pause_seconds_sum=8, go_memory_classes_heap_objects_bytes=15, node_network_receive_errs_total=28, observer_ulimit_max_fd_count=13, ob_plan_cache_access_total=18, ob_tenant_server_row_count=22, monagent_oceanbase_plan_cache_size=4, ob_tenant_server_data_size=22, monagent_oceanbase_audit_sample_cache_size=4, go_gc_heap_frees_by_size_bytes_total_sum=15, ob_sysstat_40030=19, node_load1=24, process_cpu_seconds_total=15, ob_sysstat_120001=19, go_memory_classes_total_bytes=15, ob_sysstat_120002=19, ob_sysstat_120000=19, ob_tenant_resource_max_memory=18, ob_sysstat_120009=19, go_memstats_stack_inuse_bytes=15, ob_sysstat_120008=19, node_load5=24, ob_sysstat_120005=19, go_memory_classes_os_stacks_bytes=15, ob_sysstat_120006=19, jvm_threads_peak_threads=3, monagent_input_sql_duration_seconds_count=55, ob_tenant_context_memory_hold_bytes=841, obproxy_memory_limit_bytes=7, ob_compaction_last_version=19, ob_sysstat_30000=19, ob_server_resource_cpu_assigned=13, ob_sysstat_30002=19, ob_sysstat_30001=19, monagent_oceanbase_plan_timeinfo_cache_size=4, monagent_plugin_execute_total=39, ob_sysstat_30006=19, ob_sysstat_30005=19, ob_sysstat_30008=19, ob_sysstat_30007=19, ob_sysstat_30009=19, go_gc_heap_frees_by_size_bytes_total_bucket=27, obproxy_client_max_connections=7, ob_compaction_current_timestamp=19, ob_clog_max_sync_delay_seconds=20, go_gc_pauses_seconds_total_sum=15, ob_compaction_start_time=19} 2024-03-13 15:55:00.569 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect trans stat to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:55:00.569 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.569 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect trans stat to SqlAggregateData for alarm, taskDefinitionId=999983, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.572 INFO 1433467 --- [pool-prometheus-minute-schedule-1,fbce573c7eb84c5f,f6cfe8b80370] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=18, collectTaskCount=18 2024-03-13 15:55:00.572 INFO 1433467 --- [pool-prometheus-minute-schedule-1,fbce573c7eb84c5f,f6cfe8b80370] c.o.ocp.monitor.OcpMonitorManager : Schedule interval metric too slow, from=1710316500, count=18, elapsed=1708606184072 2024-03-13 15:55:00.580 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.581 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999984, name=Get agent alarms, templateName=GetAgentAlarmsSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:54:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.587 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Get agent alarms, nodeCount=1, taskCount=1 2024-03-13 15:55:00.587 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.587 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Get agent alarms, taskDefinitionId=999984, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.591 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:50:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.591 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999985, name=SqlParseSchedule, templateName=SqlParseSchedule, scheduleRule=0 */5 * * * *, lastRunTime=2024-03-13 15:50:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.601 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.p.s.internal.SqlParseTaskSchedule : get Clusters for parse:[1, 2] 2024-03-13 15:55:00.611 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.p.s.internal.SqlParseTaskSchedule : HighRiskSql Parse is ready 2024-03-13 15:55:00.611 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=SqlParseTaskSchedule, nodeCount=1, taskCount=1 2024-03-13 15:55:00.611 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.611 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=SqlParseTaskSchedule, taskDefinitionId=999985, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.616 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:50:02.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.616 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999986, name=Check all cluster configUrl, templateName=CheckClusterConfigUrlSchedule, scheduleRule=0 */5 * * * ?, lastRunTime=2024-03-13 15:50:02.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.618 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=CheckClusterConfigUrlSchedule, nodeCount=1, taskCount=1 2024-03-13 15:55:00.618 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.618 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=CheckClusterConfigUrlSchedule, taskDefinitionId=999986, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.625 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:31.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.625 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:54:31.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.626 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:55:00.626 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.626 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.738 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016732, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.753 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016733, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.756 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016732, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014383, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.734+08:00, sub_task_instance_name=Detect alarms of backup, sub_task_instance_id=10016732}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.764 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016734, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.767 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016733, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014384, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.751+08:00, sub_task_instance_name=Update diag function, sub_task_instance_id=10016733}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.777 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016735, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.777 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.778 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016734, context=Context{parallelIdx=0, stringMap={subtask_splitter=tenant_ids, task_instance_id=10014385, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.762+08:00, sub_task_instance_name=High Risk Sql Audit Stat Task, sub_task_instance_id=10016734}, listMap={cluster_ids=[[1,2]], tenant_ids=[[2,4]]}}, executor=10.10.10.148 2024-03-13 15:55:00.779 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.t.HighRiskSqlAuditStatTask : start to diagnose[2, 4] 2024-03-13 15:55:00.778 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:00.780 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.786 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.788 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:55:00.793 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_1_2currentTimeUs:1710316500779000 2024-03-13 15:55:00.794 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_1_2diagnosisStartTimeUs:1710316201199000 2024-03-13 15:55:00.805 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,41d527ac6c6c] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016735, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014386, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.775+08:00, sub_task_instance_name=Collect all tenants mini compactions, sub_task_instance_id=10016735}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.805 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,41d527ac6c6c] t.b.t.CollectAllTenantMiniCompactionTask : Begin to collect all tenants mini compaction info 2024-03-13 15:55:00.808 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016736, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.818 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016737, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.818 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect mini compaction info 2024-03-13 15:55:00.818 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect mini compaction info 2024-03-13 15:55:00.822 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,6fbd46b119bd] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016736, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014387, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.806+08:00, sub_task_instance_name=Collect all tenants major compactions, sub_task_instance_id=10016736}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.822 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,6fbd46b119bd] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2024-03-13 15:55:00.825 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016738, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.825 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_1_2totalLength:0 2024-03-13 15:55:00.828 INFO 1433467 --- [pool-subtask-executor14,f47a761474a84dce,5182fb0b3024] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016737, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014388, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.816+08:00, sub_task_instance_name=Collect SlowSql to SqlAggregateData for alarm, sub_task_instance_id=10016737}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.829 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_2_4currentTimeUs:1710316500827000 2024-03-13 15:55:00.829 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_2_4diagnosisStartTimeUs:1710316201173000 2024-03-13 15:55:00.833 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:55:00.833 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:55:00.833 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:55:00.833 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:55:00.834 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_2_4totalLength:0 2024-03-13 15:55:00.836 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016739, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.839 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:00.839 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.839 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.840 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,a289468e6e06] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016738, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014389, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.823+08:00, sub_task_instance_name=Collect trans stat to SqlAggregateData for alarm, sub_task_instance_id=10016738}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.840 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.840 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.841 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:55:00.841 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.841 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.842 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.842 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.842 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:00.842 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:55:00.842 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.843 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.843 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.843 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016734, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.844 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:55:00.845 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402011840869, broadcast scn:1710266402011840869, saved frozen scn:1710266402011840869 2024-03-13 15:55:00.845 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:55:00.846 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266403004770377, broadcast scn:1710266403004770377, saved frozen scn:1710266403004770377 2024-03-13 15:55:00.847 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:55:00.848 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266400687471944, broadcast scn:1710266400687471944, saved frozen scn:1710266400687471944 2024-03-13 15:55:00.848 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:55:00.849 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.850 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.850 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.850 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402269282986, broadcast scn:1710266402269282986, saved frozen scn:1710266402269282986 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016740, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, MIN(unix_timestamp(`start_time`)*1000000) AS min_start_time, MAX(unix_timestamp(`finish_time`)*1000000) AS max_finish_time, SUM(occupy_size) AS occupy_size, SUM(total_row_count) AS total_row_count, COUNT(1) AS tablet_count FROM GV$OB_TABLET_COMPACTION_HISTORY WHERE type = 'MINI_MERGE' AND unix_timestamp(`finish_time`)*1000000 > ? AND unix_timestamp(`finish_time`)*1000000 <= ? GROUP BY tenant_id;, args: [1710315300000000, 1710316500000000] 2024-03-13 15:55:00.853 INFO 1433467 --- [pool-subtask-executor29,f47a761474a84dce,48006746c3a2] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016739, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014390, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.831+08:00, sub_task_instance_name=Get all agent alarms, sub_task_instance_id=10016739}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.855 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,6fbd46b119bd] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2024-03-13 15:55:00.858 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,a289468e6e06] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster1, trans stat result is empty. 2024-03-13 15:55:00.858 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,a289468e6e06] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster, trans stat result is empty. 2024-03-13 15:55:00.858 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,6fbd46b119bd] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016736, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.858 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.858 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.859 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, MIN(unix_timestamp(`start_time`)*1000000) AS min_start_time, MAX(unix_timestamp(`finish_time`)*1000000) AS max_finish_time, SUM(occupy_size) AS occupy_size, SUM(total_row_count) AS total_row_count, COUNT(1) AS tablet_count FROM GV$OB_TABLET_COMPACTION_HISTORY WHERE type = 'MINI_MERGE' AND unix_timestamp(`finish_time`)*1000000 > ? AND unix_timestamp(`finish_time`)*1000000 <= ? GROUP BY tenant_id;, args: [1710315300000000, 1710316500000000] 2024-03-13 15:55:00.860 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016741, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.870 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,a289468e6e06] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016738, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.871 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016742, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.876 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016740, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014391, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.850+08:00, sub_task_instance_name=Sql Parse Task, sub_task_instance_id=10016740}, listMap={cluster_ids=[[1,2]]}}, executor=10.10.10.148 2024-03-13 15:55:00.876 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.p.sql.internal.task.SqlParseTask : parse sql for cluster:1 2024-03-13 15:55:00.879 INFO 1433467 --- [pool-subtask-executor14,f47a761474a84dce,5182fb0b3024] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for obcluster1, save 1 to SqlAggregateData 2024-03-13 15:55:00.881 INFO 1433467 --- [pool-subtask-executor14,f47a761474a84dce,5182fb0b3024] c.o.o.p.s.i.alarm.SlowSqlDataSource : query sqlCount of obCluster obcluster, sql count is 0. 2024-03-13 15:55:00.881 INFO 1433467 --- [pool-subtask-executor14,f47a761474a84dce,5182fb0b3024] c.o.o.p.s.i.alarm.SlowSqlDataSource : query SlowSqlData of obCluster obcluster, result is empty. 2024-03-13 15:55:00.882 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,894e0c37e2cb] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016741, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014392, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.858+08:00, sub_task_instance_name=Check root service , sub_task_instance_id=10016741}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.882 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:00.883 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,894e0c37e2cb] c.o.o.s.t.b.c.AbstractAllClusterTask : Check root service executed completed, tryCount=2, successCount=0 2024-03-13 15:55:00.883 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect mini compaction info 2024-03-13 15:55:00.920 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.884 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,02cc2db91f20] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016742, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014393, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.870+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016742}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.923 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.926 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.928 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:55:00.930 INFO 1433467 --- [pool-Check obproxy10,f47a761474a84dce,fefdadc2e595] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:55:00.930 INFO 1433467 --- [pool-Check obproxy5,f47a761474a84dce,5ce32a29443d] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:55:00.930 INFO 1433467 --- [pool-Check obproxy15,f47a761474a84dce,d2d852e369d7] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:55:00.932 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,93b21b0dfde5] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:55:00.933 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2024-03-13 15:55:00.933 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.934 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.934 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.934 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [obconfig_url] 2024-03-13 15:55:00.934 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1,obcluster1,1002,1710288000000000,2000,30000000 2024-03-13 15:55:00.885 INFO 1433467 --- [pool-subtask-executor14,f47a761474a84dce,5182fb0b3024] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016737, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.920 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect mini compaction info 2024-03-13 15:55:00.935 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2024-03-13 15:55:00.935 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2024-03-13 15:55:00.923 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,894e0c37e2cb] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016741, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.924 INFO 1433467 --- [pool-subtask-executor29,f47a761474a84dce,48006746c3a2] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016739, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.936 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:00.938 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:00.939 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:00.939 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:00.941 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.perf.sql.internal.SqlParseService : parse 0 sqls for tenant:ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL} 2024-03-13 15:55:00.941 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.p.sql.internal.task.SqlParseTask : parse sql for cluster:2 2024-03-13 15:55:00.941 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:00.941 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:00.942 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.943 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.943 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.943 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [obconfig_url] 2024-03-13 15:55:00.944 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=6 2024-03-13 15:55:00.944 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:55:00.950 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:00.950 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=7 2024-03-13 15:55:00.955 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,41d527ac6c6c] t.b.t.CollectAllTenantMiniCompactionTask : Finish to collect all tenants mini compaction info 2024-03-13 15:55:00.955 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2024-03-13 15:55:00.955 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2024-03-13 15:55:00.956 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,e1458fdc5db148cb,1d87c9b6489f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=obcluster1&ObClusterId=1&version=2, client=10.10.10.148, traceId=e1458fdc5db148cb, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getRootServiceInfo(String,Long,String,Long,boolean,Integer), args=obcluster1,1,false,2,] 2024-03-13 15:55:00.958 INFO 1433467 --- [pool-Check obproxy5,f47a761474a84dce,5ce32a29443d] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:55:00.958 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,e1458fdc5db148cb,1d87c9b6489f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=e1458fdc5db148cb, duration=3 ms] 2024-03-13 15:55:00.959 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.948 INFO 1433467 --- [pool-Check obproxy10,f47a761474a84dce,fefdadc2e595] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:55:00.956 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-195,2509f16a1b404931,85a05963a38b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=obcluster&ObClusterId=1&version=2, client=10.10.10.148, traceId=2509f16a1b404931, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getRootServiceInfo(String,Long,String,Long,boolean,Integer), args=obcluster,1,false,2,] 2024-03-13 15:55:00.959 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.959 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.959 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:00.960 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-195,2509f16a1b404931,85a05963a38b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=2509f16a1b404931, duration=5 ms] 2024-03-13 15:55:00.960 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:00.966 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.966 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,41d527ac6c6c] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016735, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.966 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.966 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [rootservice_list] 2024-03-13 15:55:00.966 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:00.969 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [rootservice_list] 2024-03-13 15:55:00.971 INFO 1433467 --- [pool-Check obproxy15,f47a761474a84dce,d2d852e369d7] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:55:00.971 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:00.972 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.972 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:00.972 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.972 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.10.10.148, 2882] 2024-03-13 15:55:00.973 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1,obcluster,1002,1710288000000000,2000,30000000 2024-03-13 15:55:00.973 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:00.973 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:00.974 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.perf.sql.internal.SqlParseService : parse 0 sqls for tenant:ObTenantEntity{id=4, name=ocp, obTenantId=1002, status=NORMAL} 2024-03-13 15:55:00.977 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.979 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016740, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.979 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:00.980 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.980 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.980 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.980 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.10.10.145, 2882] 2024-03-13 15:55:00.981 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,93b21b0dfde5] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:55:00.981 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.981 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.982 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.984 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.984 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.10.10.146, 2882] 2024-03-13 15:55:00.982 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.982 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.984 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.986 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.986 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.987 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.988 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,02cc2db91f20] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016742, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.988 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.10.10.147, 2882] 2024-03-13 15:55:00.988 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:55:00.994 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:01.004 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014385, status:SUCCESS 2024-03-13 15:55:01.004 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014385, status:SUCCESS 2024-03-13 15:55:01.009 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014386, status:SUCCESS 2024-03-13 15:55:01.009 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014386, status:SUCCESS 2024-03-13 15:55:01.013 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014387, status:SUCCESS 2024-03-13 15:55:01.014 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014387, status:SUCCESS 2024-03-13 15:55:01.017 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.o.DiskUsageTimeRemainingOperator : do least Square for obcluster1 2024-03-13 15:55:01.017 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.ocp.diagnosis.model.NeuralNetwork : W:[566.8803467723029], b:[-9.491477515815789E11] 2024-03-13 15:55:01.018 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014388, status:SUCCESS 2024-03-13 15:55:01.019 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014388, status:SUCCESS 2024-03-13 15:55:01.019 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.022 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.023 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014389, status:SUCCESS 2024-03-13 15:55:01.024 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.024 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014389, status:SUCCESS 2024-03-13 15:55:01.026 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.031 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014390, status:SUCCESS 2024-03-13 15:55:01.027 INFO 1433467 --- [pool-scanner-1609793012-1,f47a761474a84dce,ca8d4c673b74] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.031 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.031 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014390, status:SUCCESS 2024-03-13 15:55:01.033 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:55:01.037 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014391, status:SUCCESS 2024-03-13 15:55:01.037 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014391, status:SUCCESS 2024-03-13 15:55:01.038 INFO 1433467 --- [pool-scanner-1609793013-1,f47a761474a84dce,916c0e02707a] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.040 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.043 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:55:01.047 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014392, status:SUCCESS 2024-03-13 15:55:01.049 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014392, status:SUCCESS 2024-03-13 15:55:01.053 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014393, status:SUCCESS 2024-03-13 15:55:01.053 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014393, status:SUCCESS 2024-03-13 15:55:01.058 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.061 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.071 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.073 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.080 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.081 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.083 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:55:01.084 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:55:01.087 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.088 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.090 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [3600] 2024-03-13 15:55:01.103 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.120 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:01.120 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:01.122 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=3600, delayTasks=[] 2024-03-13 15:55:01.131 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.133 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.144 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.146 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.148 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.149 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.151 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:55:01.152 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:55:01.153 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.155 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.156 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [300] 2024-03-13 15:55:01.173 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=300, delayTasks=[] 2024-03-13 15:55:01.188 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.190 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.192 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.193 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.195 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [90] 2024-03-13 15:55:01.224 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=90, timeoutTasks=[] 2024-03-13 15:55:01.231 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.233 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.234 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.238 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.241 INFO 1433467 --- [pool-scanner-1609793048-1,f47a761474a84dce,1645838e796b] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.242 INFO 1433467 --- [pool-scanner-1609793049-1,f47a761474a84dce,ed8bc8e33f83] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=48 2024-03-13 15:55:01.242 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.245 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [60] 2024-03-13 15:55:01.251 INFO 1433467 --- [pool-scanner-1609793051-1,f47a761474a84dce,b67e3a85a04c] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=48 2024-03-13 15:55:01.251 INFO 1433467 --- [pool-scanner-1609793050-1,f47a761474a84dce,289060e945cd] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.255 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.260 INFO 1433467 --- [pool-scanner-1609793053-1,f47a761474a84dce,27a66a065ff6] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.264 INFO 1433467 --- [pool-scanner-1609793052-1,f47a761474a84dce,b8c11b34feb7] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.267 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=60, timeoutTasks=[] 2024-03-13 15:55:01.274 INFO 1433467 --- [pool-scanner-1609793055-1,f47a761474a84dce,6c8e065f171c] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.275 INFO 1433467 --- [pool-scanner-1609793054-1,f47a761474a84dce,cf7bce6364b5] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.279 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.279 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.281 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.282 WARN 1433467 --- [pool-scanner-1609793057-1,f47a761474a84dce,15327d2ee038] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = ob_tenant_disk_total_size, seriesId size = 16 2024-03-13 15:55:01.283 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.287 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.290 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:55:01.294 INFO 1433467 --- [pool-scanner-1609793057-1,f47a761474a84dce,15327d2ee038] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=16, nodesFilledCount=32 2024-03-13 15:55:01.314 INFO 1433467 --- [pool-scanner-1609793059-1,f47a761474a84dce,9d3be4dac064] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=16, nodesFilledCount=32 2024-03-13 15:55:01.320 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.334 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:55:01.342 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.344 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.345 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.347 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.350 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:55:01.371 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.374 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:55:01.377 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster1 does not enable secondary backup 2024-03-13 15:55:01.379 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster does not enable secondary backup 2024-03-13 15:55:01.383 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackup/], metricValues size:[0]. 2024-03-13 15:55:01.385 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackupocp/], metricValues size:[0]. 2024-03-13 15:55:01.392 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016732, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:01.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.o.DiskUsageTimeRemainingOperator : do least Square for obcluster 2024-03-13 15:55:01.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.ocp.diagnosis.model.NeuralNetwork : W:[238.19327327213082], b:[-3.98276346274587E11] 2024-03-13 15:55:01.395 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.400 INFO 1433467 --- [pool-scanner-1609793074-1,f47a761474a84dce,52e91b10a0b0] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.408 INFO 1433467 --- [pool-scanner-1609793075-1,f47a761474a84dce,33938c408303] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.410 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.450 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.507 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.509 INFO 1433467 --- [pool-scanner-1609793110-1,f47a761474a84dce,19447b7a4ac5] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.511 INFO 1433467 --- [pool-scanner-1609793111-1,f47a761474a84dce,a83dcbce334a] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=7 2024-03-13 15:55:01.518 INFO 1433467 --- [pool-scanner-1609793112-1,f47a761474a84dce,786e357c943b] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.518 INFO 1433467 --- [pool-scanner-1609793113-1,f47a761474a84dce,eeceb8a8109f] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=7 2024-03-13 15:55:01.521 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.523 INFO 1433467 --- [pool-scanner-1609793114-1,f47a761474a84dce,7f6ee13f7cf3] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.525 INFO 1433467 --- [pool-scanner-1609793115-1,f47a761474a84dce,013900610e7a] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.531 INFO 1433467 --- [pool-scanner-1609793117-1,f47a761474a84dce,130981a6c3db] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.532 INFO 1433467 --- [pool-scanner-1609793116-1,f47a761474a84dce,86ec9e46b791] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.534 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.543 INFO 1433467 --- [pool-scanner-1609793119-1,f47a761474a84dce,2df9d374d04e] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=16, nodesFilledCount=32 2024-03-13 15:55:01.559 INFO 1433467 --- [pool-scanner-1609793121-1,f47a761474a84dce,c3b93f9fceca] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=16, nodesFilledCount=32 2024-03-13 15:55:01.573 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016733, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:01.891 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804715,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:02.081 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014383, status:SUCCESS 2024-03-13 15:55:02.082 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014383, status:SUCCESS 2024-03-13 15:55:02.092 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014384, status:SUCCESS 2024-03-13 15:55:02.093 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014384, status:SUCCESS 2024-03-13 15:55:02.146 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,f5b7a114c7b64578,7ceb16e6b8a7] c.o.o.s.c.trace.RequestTracingAspect : API: [POST /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=obcluster1&version=2&ObClusterId=1, client=10.10.10.146, traceId=f5b7a114c7b64578, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.updateRootServiceInfo(String,Long,String,Long,Integer,String), args=obcluster1,1,2,{"ObRegion":"obcluster1","ObCluster":"obcluster1","ObRegionId":1,"ObClusterId":1,"Type":"PRIMARY","timestamp":1710316502144288,"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"ReadonlyRsList":[]},] 2024-03-13 15:55:02.159 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,f5b7a114c7b64578,7ceb16e6b8a7] c.o.ocp.configurl.ConfigUrlService : updateRootServer, clusterName=obcluster1, clusterId=1, affectRows=1 2024-03-13 15:55:02.160 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,f5b7a114c7b64578,7ceb16e6b8a7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [POST /services client=10.10.10.146, traceId=f5b7a114c7b64578, duration=15 ms] 2024-03-13 15:55:03.892 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804738,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:04.120 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:04.121 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:05.892 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804761,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:06.104 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804765,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:06.749 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:06.748755+08:00, lastProcessTime=2024-03-13T15:54:56.745596+08:00 2024-03-13 15:55:06.863 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:06.748755+08:00 2024-03-13 15:55:06.923 INFO 1433467 --- [pool-ocp-schedules-2,79505fbdec8440bf,29721cf315a4] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:07.121 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:07.122 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:07.865 INFO 1433467 --- [pool-15-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : Starting PLAN_CHANGE_DETECTOR, Interval{start=2024-03-13T15:51:07.61, end=2024-03-13T15:53:07.863} 2024-03-13 15:55:07.873 INFO 1433467 --- [pool-15-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR : for [Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null)] 2024-03-13 15:55:07.874 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:55:07.878 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:07.61, end=2024-03-13T15:53:07.863},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:55:07.878 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:55:07.878 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:55:07.887 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:55:07.888 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 1 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:07.61, end=2024-03-13T15:53:07.863},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:55:07.892 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804784,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:07.894 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:55:07.894 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:55:07.948 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107948000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=C3C7525B95B37E68FE1C3D3AC19F8E2B, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.949 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107949000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=B7A34188E00F96CA660B2D39A3968328, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.949 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107949000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.951 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107951000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.951 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107951000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=DD3D1866D187B8E0DC29B621473BC036, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.959 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 233 2024-03-13 15:55:07.959 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.961 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107961000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=C93AE033743B2D4CAB9F14399C625DAF, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.962 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2024-03-13 15:55:07.962 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.965 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107965000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=E204274999A63B8CBD67199F075C5D97, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.966 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 89 2024-03-13 15:55:07.966 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.966 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 200 2024-03-13 15:55:07.966 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.967 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 382 2024-03-13 15:55:07.967 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.967 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107967000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=ED37973188EACB829E6683C582C7E117, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.968 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 274 2024-03-13 15:55:07.968 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.970 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.970 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=B7A34188E00F96CA660B2D39A3968328, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.973 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.973 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=DD3D1866D187B8E0DC29B621473BC036, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=C93AE033743B2D4CAB9F14399C625DAF, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 258 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.977 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.977 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.979 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.979 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.981 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.981 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=C3C7525B95B37E68FE1C3D3AC19F8E2B, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.982 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.982 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.984 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.984 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.984 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 245 2024-03-13 15:55:07.984 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.985 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.985 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.985 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.985 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=E204274999A63B8CBD67199F075C5D97, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.986 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.986 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=C93AE033743B2D4CAB9F14399C625DAF, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.987 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.987 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.987 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 6 records,use plan union hash 2024-03-13 15:55:07.987 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.987 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=DD3D1866D187B8E0DC29B621473BC036, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.989 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:55:07.993 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.961, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=C93AE033743B2D4CAB9F14399C625DAF, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:07.993 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:55:07.993 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.993 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.994 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.994 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=B7A34188E00F96CA660B2D39A3968328, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.994 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.994 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=C3C7525B95B37E68FE1C3D3AC19F8E2B, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.995 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.951, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=DD3D1866D187B8E0DC29B621473BC036, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:07.996 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:55:07.996 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.996 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=E204274999A63B8CBD67199F075C5D97, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.997 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.949, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=B7A34188E00F96CA660B2D39A3968328, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:07.997 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:55:07.997 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2024-03-13 15:55:07.998 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.998 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=ED37973188EACB829E6683C582C7E117, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.999 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.948, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=C3C7525B95B37E68FE1C3D3AC19F8E2B, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:07.999 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.951, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:07.999 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 6 records,use plan union hash 2024-03-13 15:55:08.000 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:08.000 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:08.000 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.965, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=E204274999A63B8CBD67199F075C5D97, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:08.006 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 4 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.949, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:08.008 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:08.008 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=ED37973188EACB829E6683C582C7E117, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:08.010 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 4 records,use plan union hash 2024-03-13 15:55:08.013 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.967, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=ED37973188EACB829E6683C582C7E117, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:08.013 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:55:08.014 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:07.61, end=2024-03-13T15:53:07.863},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:55:08.014 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:55:08.014 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:55:08.046 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230108046000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=1, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=1, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:08.074 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 788 2024-03-13 15:55:08.074 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:08.084 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:08.084 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1, obServerId=null, obDbId=201001, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:08.086 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:08.087 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:08.095 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:08.095 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1, obServerId=null, obDbId=201001, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:08.098 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2024-03-13 15:55:08.101 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:08.046, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=1, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:08.101 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:55:08.102 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:07.61, end=2024-03-13T15:53:07.863},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:55:08.102 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:55:08.102 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:55:08.103 INFO 1433467 --- [pool-15-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR, Found out 0 dubious sqls 2024-03-13 15:55:09.248 INFO 1433467 --- [pool-ocp-schedules-11,ea026846b4204ce0,d89131997ff0] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2024-03-13 15:55:09.255 INFO 1433467 --- [pool-ocp-schedules-15,c0140cd69d8b4bae,0ddea33cf400] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:55:09.258 INFO 1433467 --- [pool-ocp-schedules-13,c629d149f6af4d69,152121145329] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:55:09.258 INFO 1433467 --- [pool-ocp-schedules-15,c0140cd69d8b4bae,0ddea33cf400] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:55:09.261 INFO 1433467 --- [pool-ocp-schedules-13,c629d149f6af4d69,152121145329] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:55:09.263 INFO 1433467 --- [pool-ocp-schedules-11,ea026846b4204ce0,d89131997ff0] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2024-03-13 15:55:09.264 INFO 1433467 --- [pool-ocp-schedules-6,20c0e83a0d1e4204,fa3100b236cd] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:55:09.266 INFO 1433467 --- [pool-ocp-schedules-6,20c0e83a0d1e4204,fa3100b236cd] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:55:09.267 INFO 1433467 --- [pool-ocp-schedules-12,b3c7547a73e54993,e8846c42b96e] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:55:09.267 INFO 1433467 --- [pool-Check host task for clockdiff25,b3c7547a73e54993,f3124660a200] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff19,b3c7547a73e54993,987ad4c0ca86] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff13,b3c7547a73e54993,17efb8e46ec0] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff11,b3c7547a73e54993,218de21a2ef9] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff26,b3c7547a73e54993,5bcd9b7395f0] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff22,b3c7547a73e54993,865b6468034c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff14,b3c7547a73e54993,8add1b662569] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-ocp-schedules-10,aedc047512554686,2224b5a4936e] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-ocp-schedules-14,416573bd302c485b,b679f86b2c5a] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-ocp-schedules-17,eaff7f3ddae4496f,0e38ee71e4b8] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-ocp-schedules-4,6ade802bff964a9d,135af58984c8] c.o.o.s.task.scheduled.ClusterSchedules : Check host version schedule. 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-ocp-schedules-4,6ade802bff964a9d,135af58984c8] o.o.c.t.e.u.DistributedExclusiveExecutor : Empty item list, taskName=Check host task for host version, skip execute 2024-03-13 15:55:09.320 WARN 1433467 --- [pool-Check host task for clockdiff26,b3c7547a73e54993,5bcd9b7395f0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.320 ERROR 1433467 --- [pool-Check host task for clockdiff26,b3c7547a73e54993,5bcd9b7395f0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:55:09.320 WARN 1433467 --- [pool-Check host task for clockdiff26,b3c7547a73e54993,5bcd9b7395f0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.325 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:09.325 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:09.326 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:09.328 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:09.328 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:09.328 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:09.331 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:09.332 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:09.333 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:09.334 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:09.335 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:09.335 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:09.336 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804801,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:09.338 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:09.340 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:09.354 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.354 INFO 1433467 --- [pool-ocp-schedules-3,9c31d9ee96994c77,c945be7d14d3] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:55:09.355 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:55:09.355 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.355 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.357 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:55:09.357 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:55:09.357 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:55:09.357 WARN 1433467 --- [pool-Check host task for clockdiff14,b3c7547a73e54993,8add1b662569] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.357 WARN 1433467 --- [pool-Check host task for clockdiff25,b3c7547a73e54993,f3124660a200] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.358 ERROR 1433467 --- [pool-Check host task for clockdiff14,b3c7547a73e54993,8add1b662569] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:55:09.358 ERROR 1433467 --- [pool-Check host task for clockdiff25,b3c7547a73e54993,f3124660a200] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:55:09.358 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:09.358 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:55:09.358 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:55:09.358 WARN 1433467 --- [pool-Check host task for clockdiff25,b3c7547a73e54993,f3124660a200] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.358 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:55:09.359 WARN 1433467 --- [pool-Check host task for clockdiff19,b3c7547a73e54993,987ad4c0ca86] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.359 ERROR 1433467 --- [pool-Check host task for clockdiff19,b3c7547a73e54993,987ad4c0ca86] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:55:09.359 WARN 1433467 --- [pool-Check host task for clockdiff19,b3c7547a73e54993,987ad4c0ca86] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.358 WARN 1433467 --- [pool-Check host task for clockdiff14,b3c7547a73e54993,8add1b662569] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.363 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:09.363 INFO 1433467 --- [ForkJoinPool-1386568-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.363 INFO 1433467 --- [ForkJoinPool-1386568-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.363 INFO 1433467 --- [ForkJoinPool-1386568-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.364 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:09.364 INFO 1433467 --- [ForkJoinPool-1386568-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:09.364 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:09.364 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:09.366 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:09.366 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:09.366 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:09.367 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:09.367 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:09.367 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:09.367 INFO 1433467 --- [ForkJoinPool-1386568-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:55:09.367 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:09.368 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:09.368 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:09.368 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:09.369 INFO 1433467 --- [ForkJoinPool-1386568-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.369 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:55:09.369 INFO 1433467 --- [ForkJoinPool-1386568-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.369 INFO 1433467 --- [ForkJoinPool-1386568-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.370 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:55:09.371 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:55:09.371 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:55:09.371216023+08:00 2024-03-13 15:55:09.371 INFO 1433467 --- [ForkJoinPool-1386568-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:09.372 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:55:09.371+08:00 2024-03-13 15:55:09.372 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:55:09.373 INFO 1433467 --- [ForkJoinPool-1386568-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:55:09.373 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:55:09.372872640+08:00 2024-03-13 15:55:09.373 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:55:09.373167565+08:00 2024-03-13 15:55:09.374 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:55:09.371+08:00 and status AVAILABLE 2024-03-13 15:55:09.374 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:55:09.373+08:00 2024-03-13 15:55:09.374 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:55:09.373+08:00 2024-03-13 15:55:09.375 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:55:09.374450394+08:00 2024-03-13 15:55:09.376 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:55:09.373+08:00 and status AVAILABLE 2024-03-13 15:55:09.376 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:55:09.377 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:55:09.375+08:00 2024-03-13 15:55:09.377 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:55:09.378 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.378 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.378 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.378 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:09.379 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:55:09.375+08:00 and status AVAILABLE 2024-03-13 15:55:09.379 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:55:09.373+08:00 and status AVAILABLE 2024-03-13 15:55:09.379 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:55:09.379262241+08:00 2024-03-13 15:55:09.380 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:55:09.379+08:00 2024-03-13 15:55:09.380 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:55:09.381 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:55:09.379+08:00 and status AVAILABLE 2024-03-13 15:55:09.381 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:55:09.383 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:55:09.383 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.383 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:55:09.384 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:09.385 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:09.386 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.386 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:55:09.386 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:55:09.386 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:55:09.387 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.388 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:55:09.388 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.388 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:09.389 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:09.389 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:09.390 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:55:09.389835713+08:00 2024-03-13 15:55:09.390 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:55:09.390+08:00 2024-03-13 15:55:09.390 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:09.391 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:55:09.390+08:00 and status AVAILABLE 2024-03-13 15:55:09.391 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:55:09.391 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:55:09.391 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:55:09.393 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:09.394 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:55:09.395 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:55:09.395 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:55:09.395066471+08:00 2024-03-13 15:55:09.395 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:09.395 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:09.396 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:55:09.395+08:00 2024-03-13 15:55:09.396 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:09.400 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:55:09.395+08:00 and status AVAILABLE 2024-03-13 15:55:09.400 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:09.401 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:09.402 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.402 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.403 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.404 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.406 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:55:09.407 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:55:09.407 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:55:09.407 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:55:09.408 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.411 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:55:09.411 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:55:09.411 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:55:09.422 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.422 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.424 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:09.425 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.425 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:09.429 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.432 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:55:09.433 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:55:09.436 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:55:09.439 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:55:09.439 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:55:09.441 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.442 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:55:09.447 WARN 1433467 --- [pool-scanner-1609793253-1,86b62a8260ff47eb,74c2daedead7] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:09.449 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.451 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.451 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.455 WARN 1433467 --- [pool-Check host task for clockdiff22,b3c7547a73e54993,865b6468034c] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.455 ERROR 1433467 --- [pool-Check host task for clockdiff22,b3c7547a73e54993,865b6468034c] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:55:09.455 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:55:09.465 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:55:09.475 WARN 1433467 --- [pool-Check host task for clockdiff13,b3c7547a73e54993,17efb8e46ec0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.469 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:55:09.473 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:55:09.476 ERROR 1433467 --- [pool-Check host task for clockdiff13,b3c7547a73e54993,17efb8e46ec0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:55:09.473 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:55:09.455 WARN 1433467 --- [pool-Check host task for clockdiff11,b3c7547a73e54993,218de21a2ef9] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.475 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:55:09.476 ERROR 1433467 --- [pool-Check host task for clockdiff11,b3c7547a73e54993,218de21a2ef9] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:55:09.476 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:55:09.476 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:55:09.476 WARN 1433467 --- [pool-Check host task for clockdiff11,b3c7547a73e54993,218de21a2ef9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.476 WARN 1433467 --- [pool-Check host task for clockdiff13,b3c7547a73e54993,17efb8e46ec0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.480 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:55:09.482 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:55:09.492 WARN 1433467 --- [pool-Check host task for clockdiff22,b3c7547a73e54993,865b6468034c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.509 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:55:09.511 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:55:09.514 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:55:09.516 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:55:09.517 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:55:09.519 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:55:09.525 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:55:09.527 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.527 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.528 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.529 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.531 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.531 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.532 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.532 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.532 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:55:09.535 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.536 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.539 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.544 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:55:09.546 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:55:09.550 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:55:09.553 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.554 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.554 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.554 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.555 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:09.556 WARN 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:55:09.557 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.557 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.557 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:55:09.558 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:09.559 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:55:09.560 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.566 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.566 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:09.566 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:55:09.572 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:55:09.573 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:55:09.574 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:55:09.575 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.575 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:55:09.577 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:55:09.579 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:55:09.581 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:55:09.581 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:55:09.582 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.582 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.582 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.583 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.583 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:09.583 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:55:09.586 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:55:09.586 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:55:09.587 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:55:09.588 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:09.589 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:09.593 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:55:09.595 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.602 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:55:09.610 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.615 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.635 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:55:09.644 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:55:09.653 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:55:09.659 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:55:09.664 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:55:09.667 INFO 1433467 --- [alarm-detect-16,926f03f3a2d6463c,540d6355c939] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000911, data is empty. 2024-03-13 15:55:09.671 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:55:09.671 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:55:09.671 INFO 1433467 --- [alarm-detect-16,926f03f3a2d6463c,540d6355c939] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:55:09.678 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.679 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.679 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.679 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.680 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:55:09.680 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.680 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.681 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.681 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.681 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:55:09.684 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:55:09.684 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710315940072000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:55:09.718 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.718 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.718 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.718 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.719 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:09.723 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.723 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.724 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:55:09.726 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:09.733 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.733 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:09.742 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.742 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:09.747 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.747 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:09.754 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.754 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:09.759 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.759 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:09.768 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.780 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.780 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.780 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.780 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.781 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:09.784 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:55:09.786 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:55:09.788 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:09.789 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:09.808 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:55:09.842 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:55:09.861 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.862 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.862 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.862 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.863 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:55:09.877 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:55:09.878 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:55:09.893 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804810,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:10.123 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:10.124 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:10.400 INFO 1433467 --- [pool-async-queue-offer-interval60-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 60, queue size = 6, ruleSize = 6. 2024-03-13 15:55:10.402 INFO 1433467 --- [pool-async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 17, ruleSize = 17. 2024-03-13 15:55:10.809 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.809 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.813 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.813 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.816 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.816 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.822 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.822 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.917 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:55:10.991 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:55:10.991 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:55:10.991 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:55:10.991 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:55:11.010 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:55:11.010 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:55:11.010 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total 2024-03-13 15:55:11.010 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_route 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_server_process_request_time 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_hit 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_insert 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_update 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_response 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_sequence 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_partition 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_table 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_request 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_request 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_response 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_commit 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_delete 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_update 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_select 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_slow 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_others 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_sequence 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_large 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_prepare_send_request_time 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_success 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_select 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_delete 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_commit 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_others 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_insert 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.019 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:55:11.019 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:55:11.019 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.019 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.019 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.893 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804833,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:12.229 INFO 1433467 --- [pool-metric-archive-second-2,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316440, nodeCount=20, affectRows=20 2024-03-13 15:55:12.229 INFO 1433467 --- [pool-metric-archive-second-2,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316440, nodeCount=20, validCount=20 2024-03-13 15:55:12.251 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316440, nodeCount=1000, affectRows=1000 2024-03-13 15:55:12.253 INFO 1433467 --- [pool-metric-archive-second-5,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316440, nodeCount=1000, affectRows=1000 2024-03-13 15:55:12.256 INFO 1433467 --- [pool-metric-archive-second-5,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316440, nodeCount=1000, validCount=1000 2024-03-13 15:55:12.256 INFO 1433467 --- [pool-metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316440, nodeCount=1000, affectRows=1000 2024-03-13 15:55:12.258 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316440, nodeCount=999, validCount=999 2024-03-13 15:55:12.259 INFO 1433467 --- [pool-metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316440, nodeCount=1000, validCount=1000 2024-03-13 15:55:12.734 INFO 1433467 --- [pool-ocp-schedules-1,83737baf3ee94e07,7d6211d96660] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:55:12.753 INFO 1433467 --- [pool-ocp-schedules-1,83737baf3ee94e07,7d6211d96660] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:12.753 INFO 1433467 --- [pool-ocp-schedules-1,83737baf3ee94e07,7d6211d96660] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:13.124 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:13.128 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:13.894 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804857,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:15.102 INFO 1433467 --- [pool-prometheus-second-schedule-1,53b9a980251443db,f7667bacc563] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=23, collectTaskCount=23 2024-03-13 15:55:15.781 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,4c9ef00321e845a1,4da20a8a5e7f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.148, traceId=4c9ef00321e845a1, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:55:15.793 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,4c9ef00321e845a1,4da20a8a5e7f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=4c9ef00321e845a1, duration=12 ms] 2024-03-13 15:55:15.894 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804880,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:16.104 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804884,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:16.128 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:16.129 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:16.180 INFO 1433467 --- [pool-ocp-schedules-19,dbd2dafdcd0f497c,5f20b3993112] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":12075,"minuteWriteQueue":0,"minuteReadCacheSize":12328,"secondWriteCacheSize":2488,"seriesIdKeySize":67930,"secondWriteQueue":0,"minuteWriteCacheSize":12393} 2024-03-13 15:55:16.180 INFO 1433467 --- [pool-ocp-schedules-19,dbd2dafdcd0f497c,5f20b3993112] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{go_memory_classes_heap_stacks_bytes=15, ob_sysstat_40012=19, ob_server_resource_memory_assigned_bytes=13, ob_sysstat_40011=19, ob_sysstat_40010=19, ob_unit_num=13, ob_database_disk_required_size=35, go_memstats_mspan_sys_bytes=15, go_memstats_stack_sys_bytes=15, ob_tenant_request_queue_total_size=18, jvm_gc_memory_allocated_bytes_total=3, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=18, ob_sysstat_40019=19, observer_fd_count=13, ob_sysstat_40018=19, processor_process_metrics_total=21, jvm_gc_max_data_size_bytes=3, go_memstats_frees_total=15, sql_audit_input_collect_data_jumped_total=8, host_available=64, mysql_output_write_sql_failed_count=17, ob_database_disk_data_size=34, ob_system_event_detail_total_timeouts=763, ob_tenant_memstore_freeze_cnt=14, ob_latch_sleep_total=312, ob_tenant_unit_min_memory=25, ob_sysstat_60087=19, ob_memstore_total_bytes=20, metric_cardinality_count=12, process_resident_memory_bytes=15, jvm_threads_states_threads=8, ob_cache_size_bytes=34, ob_tenant_unit_max_memory=25, input_collect_metrics_total=26, ob_sysstat_60083=19, ob_sysstat_50039=19, ob_sysstat_50038=19, node_disk_io_time_weighted_seconds_total=33, process_open_fds=15, ob_sysstat_50037=19, process_memory_rss_bytes=33, ob_server_resource_cpu_assigned_percent=13, ob_sysstat_50030=19, go_memory_classes_profiling_buckets_bytes=15, go_memstats_heap_inuse_bytes=15, ob_sysstat_40118=19, ob_sysstat_40117=19, ob_sysstat_40116=19, node_memory_memfree_bytes=26, ob_sysstat_40005=19, process_exists=43, ob_sysstat_40004=19, ob_sysstat_40003=19, ob_sysstat_40002=19, ob_sysstat_40001=19, go_gc_duration_seconds=20, go_memory_classes_metadata_mspan_inuse_bytes=15, ob_server_resource_disk_bytes=13, ob_sysstat_40000=19, odp_current_session=26, process_thread_count=33, ob_sysstat_50040=19, ob_sysstat_40009=19, ob_sysstat_40008=19, ob_disk_free_bytes=13, node_filesystem_avail_bytes=72, ob_sysstat_40007=19, ob_sysstat_40006=19, ob_backup_storage_capacity_bytes=650, ob_unit_config_max_iops=21, ob_plan_cache_memory_bytes=18, node_network_transmit_errs_total=28, go_gc_duration_seconds_count=15, monagent_oceanbase_plan_explain_size=4, ob_sysstat_50011=19, ob_sysstat_50010=19, node_disk_read_bytes_total=33, ob_compaction_is_suspended=19, mysql_output_write_sql_milliseconds_summary_count=17, go_memstats_lookups_total=15, mysql_output_write_sql_count=17, process_cpu_percent=33, ob_sysstat_60057=19, host_agent_process_status=18, ob_server_resource_cpu=13, ob_tenant_server_data_index_row_count=20, go_memstats_alloc_bytes=15, go_sched_gomaxprocs_threads=15, node_cpu_seconds_total_native=38, ob_sysstat_140002=19, ob_sysstat_140005=19, ob_sysstat_140006=19, ob_sysstat_140003=19, ob_tenant_resource_min_memory=18, ob_memstore_freeze_times=20, ob_sysstat_50022=19, jvm_gc_pause_seconds_count=8, ob_sysstat_50021=19, jvm_gc_overhead_percent=3, go_memory_classes_heap_unused_bytes=15, ob_cluster_status_check=13, jvm_memory_committed_bytes=10, node_net_bandwidth_bps=29, monagent_oceanbase_db_pool_idle=4, ob_tenant_server_data_index_data_size=20, go_gc_heap_allocs_by_size_bytes_total_count=15, go_memstats_heap_idle_bytes=15, go_gc_cycles_forced_gc_cycles_total=15, go_gc_pauses_seconds_total_bucket=26, ob_server_resource_memory_bytes=13, ob_tenant_assigned_mem_assigned=18, ob_tenant_memstore_minor_freeze_times=14, ob_tenant_server_required_size=22, ob_sysstat_60031=19, process_fd_count=33, ob_tenant_unit_max_cpu=25, ob_latch_wait_seconds_total=312, go_memory_classes_metadata_mcache_free_bytes=15, node_memory_buffers_bytes=26, ob_sysstat_50009=19, go_memstats_heap_released_bytes=15, node_network_receive_bytes_total=28, ob_sysstat_50005=19, ob_sysstat_50008=19, ob_sysstat_50001=19, ob_sysstat_50004=19, exposer_request_latencies_count=6, ob_sysstat_50000=19, ob_session_active_num=18, mysql_output_context_timeout_discard_metrics_count=9, go_memory_classes_metadata_mcache_inuse_bytes=15, node_disk_read_time_seconds_total=33, ob_sysstat_60021=19, ob_sysstat_60020=19, ob_sysstat_60024=19, ob_compaction_frozen_version=19, ob_sysstat_60023=19, metric_count=15, ob_sysstat_60022=19, obproxysys_connectable=1556, ob_tenant_assigned_cpu_assigned=18, mysql_output_write_sql_retry_count=17, ob_tenant_unit_min_cpu=25, ob_unit_config_max_cpu=21, http_request_milliseconds_summary_count=75, monagent_oceanbase_audit_timeinfo_cache_size=4, http_request_milliseconds_summary=78, ob_latch_immediate_miss_total=312, ob_latch_spin_get_total=312, ob_log_level_count=17, monagent_pipeline_report_metrics_total=16, exposer_transferred_bytes_total=6, ob_tenant_context_memory_count=841, node_disk_reads_completed_total=33, ob_sysstat_60019=19, ob_sysstat_60003=19, ob_sysstat_60002=19, ob_sysstat_60001=19, ob_sysstat_60000=19, ob_latch_miss_total=312, go_sched_latencies_seconds_count=15, ocp_meta_db_health=5, ob_disk_total_bytes=13, go_memstats_heap_objects=15, go_memstats_mcache_inuse_bytes=15, ob_sysstat_60005=19, ob_sysstat_60004=19, go_memory_classes_other_bytes=15, ob_memstore_active_bytes=20, go_memstats_alloc_bytes_total=15, ob_tenant_server_partition_replica_count=21, process_count=34, ob_system_event_detail_total_waits=763, odp_used_connections=25, backup_storage_capacity_used_percent=8, go_gc_heap_frees_objects_total=15, go_memory_classes_metadata_mspan_free_bytes=15, ob_disk_log_used_bytes=13, ob_compaction_is_error=19, go_gc_limiter_last_enabled_gc_cycle=15, monagent_pipeline_execute_total=24, node_filesystem_files=86, process_start_time_seconds=15, go_memstats_heap_alloc_bytes=15, ob_tenant_resource_max_cpu=18, ntp_service_available=26, go_memstats_mspan_inuse_bytes=15, mysql_output_write_sql_milliseconds_summary=20, go_gc_heap_frees_by_size_bytes_total_count=15, monagent_pipeline_execute_seconds_total=24, ob_sysstat_80001=19, go_memstats_sys_bytes=15, go_memory_classes_heap_free_bytes=15, system_load_average_1m=3, monagent_oceanbase_audit_stat_cache_size=4, ob_tenant_memory_used_bytes=22, obproxy_server_max_connections=7, ob_sysstat_80002=19, go_memstats_gc_cpu_fraction=15, ob_sysstat_80003=19, monagent_oceanbase_audit_cache_size=4, ob_tenant_context_memory_used_bytes=841, ob_sysstat_80007=19, node_disk_writes_completed_total=33, ob_sysstat_80008=19, ob_sysstat_80009=19, go_sched_goroutines_goroutines=15, go_memory_classes_metadata_other_bytes=15, go_gc_cycles_total_gc_cycles_total=15, system_cpu_usage=3, process_cpu_usage=3, node_disk_written_bytes_total=33, monagent_oceanbase_ash_cache_size=4, go_goroutines=15, ob_sysstat_80023=19, exposer_request_latencies_sum=6, ob_unit_config_min_cpu=21, process_virtual_memory_max_bytes=15, go_gc_heap_frees_bytes_total=15, ob_tenant500_memory_hold_bytes=13, exposer_scrapes_total=6, go_gc_heap_allocs_by_size_bytes_total_bucket=27, ob_sysstat_80025=19, ob_sysstat_80026=19, oceanbase_connectivity=15, node_memory_sreclaimable_bytes=26, go_sched_latencies_seconds_sum=15, tcp_retrans=25, exporter_export_metrics_bytes_total=26, node_filesystem_size_bytes=72, http_server_requests_seconds_count=203, ob_compaction_frozen_time=19, monagent_input_sql_duration_seconds_sum=55, log_stream_degraded_count=18, ob_plan_cache_hit_total=18, node_filesystem_readonly=72, ob_sysstat_80040=19, ob_sysstat_80041=19, io_await=52, go_gc_heap_allocs_objects_total=15, go_memstats_mallocs_total=15, ob_unit_config_max_memory_bytes=21, go_gc_stack_starting_size_bytes=15, io_util=52, ob_sysstat_200002=19, ob_sysstat_200001=19, ob_tenant_disk_v4_data_size=18, go_sched_latencies_seconds_bucket=26, ob_tenant_assigned_cpu_total=18, ob_tenant_server_available_disk_free=19, node_disk_write_time_seconds_total=33, cpu_count=25, output_write_metrics_total=9, ob_cluster_sync=15, go_gc_heap_objects_objects=15, odp_transaction_total=16, ob_system_event_detail_time_waited_us=763, process_max_fds=15, http_status_count=20, ob_sysstat_80057=19, ocp_alarm_detect_duration_ms_count=115, go_gc_heap_allocs_by_size_bytes_total_sum=15, go_memstats_mcache_sys_bytes=15, ob_tenant_disk_v4_log_size=18, go_cgo_go_to_c_calls_calls_total=15, ocp_alarm_detect_duration_ms_sum=115, jvm_threads_live_threads=3, node_memory_cached_bytes=26, ob_table_num=16, ob_sysstat_10000=19, ob_sysstat_10006=19, ob_sysstat_10005=19, ob_sysstat_10002=19, ob_sysstat_10001=19, go_memstats_last_gc_time_seconds=15, ob_sysstat_10004=19, ob_sysstat_10003=19, monitor_exporter_avaliable=62, log_tailer_count=15, ob_server_num=25, ob_session_all_num=18, ob_system_event_time_waited=24, ob_tenant_resource_min_cpu=18, monagent_plugin_execute_seconds_total=39, go_gc_cycles_automatic_gc_cycles_total=15, go_gc_heap_allocs_bytes_total=15, go_memstats_gc_sys_bytes=15, ob_latch_get_total=312, go_info=16, log_tailer_processed_file_count=21, node_load15=24, ob_disk_log_assigned_bytes=13, go_memory_classes_heap_released_bytes=15, jvm_memory_max_bytes=10, ob_unit_config_min_iops=21, ob_sysstat_30080=19, ob_sysstat_30082=19, ob_sysstat_30081=19, ob_sysstat_30084=19, ob_sysstat_30083=19, ob_sysstat_30086=19, ob_sysstat_30085=19, ob_tenant_log_disk_total_bytes=18, ob_memstore_freeze_trigger_bytes=20, ob_waitevent_wait_seconds_total=18, ob_server_stopped_duration_seconds=15, go_memstats_next_gc_bytes=15, node_filesystem_files_free=86, ob_memtable_max_snapshot_duration_seconds=18, monagent_oceanbase_audit_unique_key_cache_size=4, process_virtual_memory_bytes=15, ob_sysstat_20002=19, net_connectivity=15, ob_sysstat_20001=19, monagent_input_sql_duration_seconds_bucket=65, ob_latch_immediate_get_total=312, node_cpu_seconds_total=32, ob_sysstat_20000=19, ob_index_error_num=13, ob_tenant_server_available_disk_total=19, ob_tenant_memory_hold_bytes=22, ob_sysstat_30066=19, go_memstats_buck_hash_sys_bytes=15, mysql_output_write_sql_milliseconds_summary_sum=17, ob_tenant_disk_total_size=18, odp_sql_request_total=53, mysql_output_write_task_size=17, exporter_export_metrics_total=26, ob_disk_log_total_bytes=13, system_cpu_count=3, ob_sysstat_130004=19, process_uptime_seconds=33, node_ntp_offset_seconds=25, go_memstats_heap_sys_bytes=15, ob_sysstat_140012=19, ob_sysstat_140013=19, jvm_gc_pause_seconds_max=8, ob_tenant_request_queue_large_size=18, ob_sysstat_130000=19, go_memstats_other_sys_bytes=15, ob_sysstat_130001=19, jvm_memory_used_bytes=10, ob_sysstat_130002=19, go_threads=15, http_request_milliseconds_summary_sum=75, http_server_requests_seconds_sum=203, ob_tenant_assigned_mem_total=18, node_memory_memtotal_bytes=26, node_network_transmit_drop_total=28, go_gc_pauses_seconds_total_count=15, exposer_request_latencies=9, ob_waitevent_wait_total=18, go_gc_heap_goal_bytes=15, ob_tenant_log_disk_used_bytes=18, jvm_gc_memory_promoted_bytes_total=3, odp_sql_cost_total=51, ob_sysstat_30011=19, ob_sysstat_30010=19, ob_sysstat_30013=19, ob_sysstat_30012=19, ob_tenant500_memory_used_bytes=13, node_network_transmit_bytes_total=28, node_network_receive_drop_total=28, go_gc_heap_tiny_allocs_objects_total=15, go_gc_duration_seconds_sum=15, ob_server_resource_memory_assigned_percent=13, jvm_gc_pause_seconds_sum=8, go_memory_classes_heap_objects_bytes=15, node_network_receive_errs_total=28, observer_ulimit_max_fd_count=13, ob_plan_cache_access_total=18, ob_tenant_server_row_count=22, monagent_oceanbase_plan_cache_size=4, ob_tenant_server_data_size=22, monagent_oceanbase_audit_sample_cache_size=4, go_gc_heap_frees_by_size_bytes_total_sum=15, ob_sysstat_40030=19, node_load1=24, process_cpu_seconds_total=15, ob_sysstat_120001=19, go_memory_classes_total_bytes=15, ob_sysstat_120002=19, ob_sysstat_120000=19, ob_tenant_resource_max_memory=18, ob_sysstat_120009=19, go_memstats_stack_inuse_bytes=15, ob_sysstat_120008=19, node_load5=24, ob_sysstat_120005=19, go_memory_classes_os_stacks_bytes=15, ob_sysstat_120006=19, jvm_threads_peak_threads=3, monagent_input_sql_duration_seconds_count=55, ob_tenant_context_memory_hold_bytes=841, obproxy_memory_limit_bytes=7, ob_compaction_last_version=19, ob_sysstat_30000=19, ob_server_resource_cpu_assigned=13, ob_sysstat_30002=19, ob_sysstat_30001=19, monagent_oceanbase_plan_timeinfo_cache_size=4, monagent_plugin_execute_total=39, ob_sysstat_30006=19, ob_sysstat_30005=19, ob_sysstat_30008=19, ob_sysstat_30007=19, ob_sysstat_30009=19, go_gc_heap_frees_by_size_bytes_total_bucket=27, obproxy_client_max_connections=7, ob_compaction_current_timestamp=19, ob_clog_max_sync_delay_seconds=20, go_gc_pauses_seconds_total_sum=15, ob_compaction_start_time=19} 2024-03-13 15:55:16.753 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:16.753034+08:00, lastProcessTime=2024-03-13T15:55:06.748755+08:00 2024-03-13 15:55:16.854 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:16.753034+08:00 2024-03-13 15:55:16.930 INFO 1433467 --- [pool-ocp-schedules-10,7b01f3efc8d54b45,38cc5dc370ee] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:17.894 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804904,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:19.129 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:19.130 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:19.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804920,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:19.286 INFO 1433467 --- [alarm-detect-11,2fb6046976de4595,d6b6e0bc8797] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000911, data is empty. 2024-03-13 15:55:19.287 INFO 1433467 --- [alarm-detect-26,d624df6835cc41b9,8b639edaa17c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000911, data is empty. 2024-03-13 15:55:19.288 INFO 1433467 --- [alarm-detect-17,d79c77ff5c2348fe,01b61a0fc032] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000911, data is empty. 2024-03-13 15:55:19.291 INFO 1433467 --- [alarm-detect-26,d624df6835cc41b9,8b639edaa17c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:55:19.291 INFO 1433467 --- [alarm-detect-11,2fb6046976de4595,d6b6e0bc8797] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:55:19.292 INFO 1433467 --- [alarm-detect-17,d79c77ff5c2348fe,01b61a0fc032] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:55:19.314 WARN 1433467 --- [pool-scanner-1609794904-1,7b80a82cb26849f1,ff798b9bb763] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:19.436 INFO 1433467 --- [alarm-detect-2,ea6085b798594005,cdda1754a253] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316485 ~ 1710316509 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316500=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:55:19.436 INFO 1433467 --- [alarm-detect-2,ea6085b798594005,cdda1754a253] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316485 ~ 1710316509 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316500=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:55:19.436 INFO 1433467 --- [alarm-detect-2,ea6085b798594005,cdda1754a253] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316485 ~ 1710316509 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316500=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:55:19.436 INFO 1433467 --- [alarm-detect-2,ea6085b798594005,cdda1754a253] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316485 ~ 1710316509 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316500=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:55:19.436 INFO 1433467 --- [alarm-detect-2,ea6085b798594005,cdda1754a253] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316485 ~ 1710316509 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316500=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:55:19.821 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,3dcf9f6bf82041f6,57efce041dde] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.143, traceId=3dcf9f6bf82041f6, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:55:19.827 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,3dcf9f6bf82041f6,57efce041dde] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.143, traceId=3dcf9f6bf82041f6, duration=7 ms] 2024-03-13 15:55:19.895 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804927,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:20.232 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:21.0, nextRun=2024-03-13 15:55:20.0, currentDate=Wed Mar 13 15:55:20 CST 2024 2024-03-13 15:55:20.233 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999996, name=Collect all cluster compaction info, templateName=CollectClusterCompactionSchedule, scheduleRule=20 * * * * *, lastRunTime=2024-03-13 15:54:21.0, nextRunTime=2024-03-13 15:55:20.0} 2024-03-13 15:55:20.239 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all cluster compaction info, nodeCount=1, taskCount=1 2024-03-13 15:55:20.239 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:20.239 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all cluster compaction info, taskDefinitionId=999996, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:21.108 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016743, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:21.127 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,901bdd6f65b3] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016743, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014394, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:21.102+08:00, sub_task_instance_name=Collect compaction for all clusters, sub_task_instance_id=10016743}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:21.128 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,901bdd6f65b3] c.o.o.s.t.b.c.AbstractAllClusterTask : Collect compaction for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:55:21.129 INFO 1433467 --- [ForkJoinPool-1386573-worker-1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:55:21.129 INFO 1433467 --- [ForkJoinPool-1386573-worker-2,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:55:21.133 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,901bdd6f65b3] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016743, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:21.462 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014394, status:SUCCESS 2024-03-13 15:55:21.467 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014394, status:SUCCESS 2024-03-13 15:55:21.895 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804950,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:22.131 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:22.131 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:23.896 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804973,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:25.132 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:25.133 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:25.218 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,d4ba6b730a97413d,15f90ecc11c6] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.144, traceId=d4ba6b730a97413d, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:55:25.224 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,d4ba6b730a97413d,15f90ecc11c6] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.144, traceId=d4ba6b730a97413d, duration=7 ms] 2024-03-13 15:55:25.896 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804996,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:26.104 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805000,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:26.212 INFO 1433467 --- [pool-Check obproxy7,01c0f21c4d4e4400,9e8f57e710c4] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:55:26.214 INFO 1433467 --- [pool-Check obproxy2,01c0f21c4d4e4400,4931be4f4582] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:55:26.216 INFO 1433467 --- [pool-Check obproxy9,01c0f21c4d4e4400,05c82419f13c] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:55:26.218 INFO 1433467 --- [pool-Check obproxy3,01c0f21c4d4e4400,79eb8c70802e] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:55:26.749 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:26.748088+08:00, lastProcessTime=2024-03-13T15:55:16.753034+08:00 2024-03-13 15:55:26.843 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:26.748088+08:00 2024-03-13 15:55:26.936 INFO 1433467 --- [pool-ocp-schedules-7,f363e302f68542f8,36f43483a4a2] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:27.897 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805020,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:28.133 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:28.133 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:28.794 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.794 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.794 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.795 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:55:28.801 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.801 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.803 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:55:28.804 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule start 2024-03-13 15:55:28.854 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING} 2024-03-13 15:55:28.875 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=2, name=obcluster, obClusterId=1, status=RUNNING} 2024-03-13 15:55:28.882 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster1:1-3) 2024-03-13 15:55:28.882 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster:1-1) 2024-03-13 15:55:28.882 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster1:1-1) 2024-03-13 15:55:28.882 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster1:1-2) 2024-03-13 15:55:28.888 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.888 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.888 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.889 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.889 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.889 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.889 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:55:28.889 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.890 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.890 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.890 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:55:28.890 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2024-03-13 15:55:28.892 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.892 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.892 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2024-03-13 15:55:28.935 INFO 1433467 --- [sql-rollup-worker-2,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=obcluster1:1, server=2, level=0->1, c=0) - source=2024-03-13T15:53:02, start=2024-03-13T15:50:02, end=2024-03-13T15:52:32, updates=114, granule=120000ms, delay=180000ms 2024-03-13 15:55:28.942 INFO 1433467 --- [sql-rollup-worker-2,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=2, level=0->1, c=1) - not enough cached interval (source=2024-03-13T15:53:02, target=2024-03-13T15:52:32, granule=120000ms, delay=30000ms) 2024-03-13 15:55:28.943 INFO 1433467 --- [sql-rollup-worker-2,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=2, level=1->2, c=0) - not enough cached interval (source=2024-03-13T15:52:32, target=2024-03-13T15:40:32, granule=600000ms, delay=720000ms) 2024-03-13 15:55:28.944 INFO 1433467 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=obcluster1:1, server=3, level=0->1, c=0) - source=2024-03-13T15:53:00, start=2024-03-13T15:50:30, end=2024-03-13T15:53:00, updates=77, granule=120000ms, delay=150000ms 2024-03-13 15:55:28.944 INFO 1433467 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=obcluster1:1, server=1, level=0->1, c=0) - source=2024-03-13T15:53:02, start=2024-03-13T15:50:02, end=2024-03-13T15:52:32, updates=244, granule=120000ms, delay=180000ms 2024-03-13 15:55:28.946 INFO 1433467 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=3, level=0->1, c=1) - not enough cached interval (source=2024-03-13T15:53:00, target=2024-03-13T15:53:00, granule=120000ms, delay=0ms) 2024-03-13 15:55:28.946 INFO 1433467 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=3, level=1->2, c=0) - not enough cached interval (source=2024-03-13T15:53:00, target=2024-03-13T15:46:00, granule=600000ms, delay=420000ms) 2024-03-13 15:55:28.947 INFO 1433467 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=1, level=0->1, c=1) - not enough cached interval (source=2024-03-13T15:53:02, target=2024-03-13T15:52:32, granule=120000ms, delay=30000ms) 2024-03-13 15:55:28.947 INFO 1433467 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=1, level=1->2, c=0) - not enough cached interval (source=2024-03-13T15:52:32, target=2024-03-13T15:43:02, granule=600000ms, delay=570000ms) 2024-03-13 15:55:28.947 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule end, elapsed=143ms, period=120000ms 2024-03-13 15:55:29.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805037,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:29.330 WARN 1433467 --- [pool-scanner-1609795526-1,2085cc503c8248fe,46e54f9a6018] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:29.417 INFO 1433467 --- [alarm-detect-4,c67c05e4702b43fb,9835c696be4f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:55:29.420 INFO 1433467 --- [alarm-detect-4,c67c05e4702b43fb,9835c696be4f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:55:29.507 INFO 1433467 --- [alarm-detect-25,664659f4441d4e98,5859d7e7b870] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316500 ~ 1710316519 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316515=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:55:29.507 INFO 1433467 --- [alarm-detect-25,664659f4441d4e98,5859d7e7b870] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316500 ~ 1710316519 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316515=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:55:29.507 INFO 1433467 --- [alarm-detect-25,664659f4441d4e98,5859d7e7b870] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316500 ~ 1710316519 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316515=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:55:29.507 INFO 1433467 --- [alarm-detect-25,664659f4441d4e98,5859d7e7b870] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316500 ~ 1710316519 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316515=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:55:29.897 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805044,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:30.569 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:31.0, nextRun=2024-03-13 15:55:30.0, currentDate=Wed Mar 13 15:55:30 CST 2024 2024-03-13 15:55:30.569 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999990, name=Sync tenant information, templateName=TenantSyncSchedule, scheduleRule=30 0/1 * * * ?, lastRunTime=2024-03-13 15:54:31.0, nextRunTime=2024-03-13 15:55:30.0} 2024-03-13 15:55:30.574 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Sync tenant information, nodeCount=1, taskCount=2 2024-03-13 15:55:30.574 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:30.574 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:30.574 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Sync tenant information, taskDefinitionId=999990, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:30.593 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:01.0, nextRun=2024-03-13 15:55:30.0, currentDate=Wed Mar 13 15:55:30 CST 2024 2024-03-13 15:55:30.593 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:55:01.0, nextRunTime=2024-03-13 15:55:30.0} 2024-03-13 15:55:30.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:55:30.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:30.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:31.134 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:31.135 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:31.214 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016744, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:31.226 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016745, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:31.231 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016744, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014395, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:31.212+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016744}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:55:31.237 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016746, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:31.256 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016745, context=Context{parallelIdx=1, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014395, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:31.224+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016745}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:55:31.257 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,1575a0611d9d] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016746, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014396, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:31.235+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016746}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:31.268 INFO 1433467 --- [pool-Check obproxy6,f47a761474a84dce,e3e5ee02674f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:55:31.270 INFO 1433467 --- [pool-Check obproxy11,f47a761474a84dce,f74dd2770a1e] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:55:31.270 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,03080f25cdd9] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:55:31.270 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,ea3babfaf05a] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:55:31.279 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.279 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:55:31.279 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.279 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.280 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:55:31.290 INFO 1433467 --- [pool-Check obproxy6,f47a761474a84dce,e3e5ee02674f] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:55:31.293 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,03080f25cdd9] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:55:31.293 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.294 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.294 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.295 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.295 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:55:31.295 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.295 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.295 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:55:31.297 INFO 1433467 --- [pool-Check obproxy11,f47a761474a84dce,f74dd2770a1e] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:55:31.298 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,ea3babfaf05a] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:55:31.300 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.300 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.301 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.303 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.303 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.304 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.307 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,1575a0611d9d] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016746, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:31.307 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.307 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.307 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.312 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.312 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.312 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.313 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.313 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.313 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.318 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.318 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.318 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.321 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.321 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.321 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.343 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.344 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.344 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.344 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.345 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:55:31.354 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.354 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.355 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.355 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.355 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:55:31.357 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.357 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.358 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.358 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.358 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:55:31.366 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.367 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.367 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.367 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.367 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.367 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:55:31.368 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.368 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.368 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.368 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:55:31.377 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.378 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.378 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.378 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.378 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:55:31.381 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.382 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.382 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.382 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.382 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:55:31.387 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.387 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.387 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.387 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.388 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:55:31.392 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.394 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=2 2024-03-13 15:55:31.394 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:55:31.398 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:55:31.404 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.404 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.404 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.404 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.405 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:55:31.412 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.413 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.414 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.414 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.414 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.414 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.415 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.415 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.415 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.415 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:55:31.417 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2024-03-13 15:55:31.418 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:55:31.428 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=4, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:55:31.430 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.431 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.431 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=4, userWhitelist=% 2024-03-13 15:55:31.432 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.432 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.432 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.433 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.433 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.433 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.439 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:55:31.440 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=3, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=3, userWhitelist=% 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=2 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.444 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016745, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:31.447 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:55:31.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2024-03-13 15:55:31.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2024-03-13 15:55:31.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:55:31.455 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016744, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:31.653 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014395, status:SUCCESS 2024-03-13 15:55:31.653 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014395, status:SUCCESS 2024-03-13 15:55:31.661 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014396, status:SUCCESS 2024-03-13 15:55:31.661 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014396, status:SUCCESS 2024-03-13 15:55:31.897 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805066,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:32.755 INFO 1433467 --- [pool-ocp-schedules-12,0f67cf35bc654167,402666aacb17] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:55:33.174 INFO 1433467 --- [pool-ocp-schedules-12,0f67cf35bc654167,402666aacb17] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:33.174 INFO 1433467 --- [pool-ocp-schedules-12,0f67cf35bc654167,402666aacb17] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:33.233 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,52ea19a335f549a5,870ff763446a] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.142, traceId=52ea19a335f549a5, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:55:33.238 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,52ea19a335f549a5,870ff763446a] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.142, traceId=52ea19a335f549a5, duration=6 ms] 2024-03-13 15:55:33.898 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805089,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:34.135 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:34.136 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:35.898 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805112,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:36.105 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805116,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:36.748 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:36.747607+08:00, lastProcessTime=2024-03-13T15:55:26.748088+08:00 2024-03-13 15:55:36.881 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:36.747607+08:00 2024-03-13 15:55:36.944 INFO 1433467 --- [pool-ocp-schedules-4,2e7cf03823f94984,a4ba3a72229b] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:37.136 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:37.137 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:37.899 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805136,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:39.256 INFO 1433467 --- [pool-ocp-schedules-4,6ce075921d0c4f07,84c86b540b65] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:55:39.257 INFO 1433467 --- [pool-ocp-schedules-11,313f4e553fa24619,9b7f1c4a0fdb] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:55:39.259 INFO 1433467 --- [pool-ocp-schedules-14,6a7971e20b4747ed,f3289b9cf23e] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:55:39.260 INFO 1433467 --- [pool-ocp-schedules-11,313f4e553fa24619,9b7f1c4a0fdb] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:55:39.261 INFO 1433467 --- [pool-ocp-schedules-14,6a7971e20b4747ed,f3289b9cf23e] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:55:39.261 INFO 1433467 --- [pool-ocp-schedules-4,6ce075921d0c4f07,84c86b540b65] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:55:39.283 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805153,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:39.288 INFO 1433467 --- [pool-ocp-schedules-7,0e3c9d5b34fc485c,7274bcfb1670] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:55:39.290 INFO 1433467 --- [pool-ocp-schedules-2,23c294bd7f1040e7,3ebc107ad1df] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:55:39.291 INFO 1433467 --- [alarm-detect-30,72245d80b10e453a,30eb70939b49] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:55:39.293 INFO 1433467 --- [alarm-detect-17,407e54aa05d5406b,207efd599445] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:55:39.293 INFO 1433467 --- [alarm-detect-11,530eaef250264b2c,a779a6fc76e7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:55:39.299 INFO 1433467 --- [alarm-detect-17,407e54aa05d5406b,207efd599445] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:55:39.300 INFO 1433467 --- [alarm-detect-30,72245d80b10e453a,30eb70939b49] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:55:39.301 INFO 1433467 --- [alarm-detect-11,530eaef250264b2c,a779a6fc76e7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:55:39.306 INFO 1433467 --- [pool-ocp-schedules-8,c8a7656cdcc94f9b,b869de4ee501] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:55:39.309 INFO 1433467 --- [pool-ocp-schedules-17,c2b88db545634026,2552a6cfdfcb] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff8,c2b88db545634026,c485b623ec2c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff3,c2b88db545634026,d1bb7f70dd99] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff29,c2b88db545634026,2d127af644d8] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff1,c2b88db545634026,544fe2e944c0] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff2,c2b88db545634026,b96c463cc429] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff24,c2b88db545634026,6e6a70bc9526] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:39.311 INFO 1433467 --- [pool-Check host task for clockdiff6,c2b88db545634026,147b8b80b94f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:55:39.312 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:39.312 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:39.312 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:39.312 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:39.313 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:39.313 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:39.313 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:39.314 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.314 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:39.314 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:39.314 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.314 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.314 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:39.314 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:39.314 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:39.315 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:39.315 INFO 1433467 --- [ForkJoinPool-1386575-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.315 INFO 1433467 --- [ForkJoinPool-1386575-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.315 INFO 1433467 --- [ForkJoinPool-1386575-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.315 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:39.317 INFO 1433467 --- [ForkJoinPool-1386575-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:39.317 INFO 1433467 --- [ForkJoinPool-1386575-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:55:39.318 INFO 1433467 --- [pool-ocp-schedules-18,22990421198c4e6f,3106c90f3332] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.320 INFO 1433467 --- [ForkJoinPool-1386575-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.320 INFO 1433467 --- [ForkJoinPool-1386575-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.320 INFO 1433467 --- [ForkJoinPool-1386575-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.320 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.320 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.321 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:39.322 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:39.323 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:39.324 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:39.324 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:39.324 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:39.325 INFO 1433467 --- [ForkJoinPool-1386575-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:39.325 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:39.326 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:39.326 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:39.329 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:39.329 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:39.330 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:39.330 INFO 1433467 --- [ForkJoinPool-1386575-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:55:39.330 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:39.330 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:39.331 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:39.334 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:55:39.335 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:55:39.336 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:55:39.336162679+08:00 2024-03-13 15:55:39.337 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:55:39.337 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:55:39.336+08:00 2024-03-13 15:55:39.338 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:55:39.337922586+08:00 2024-03-13 15:55:39.338 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:55:39.338 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:55:39.339 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:55:39.338+08:00 2024-03-13 15:55:39.339 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:55:39.339206709+08:00 2024-03-13 15:55:39.340 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:55:39.339+08:00 2024-03-13 15:55:39.340 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:55:39.336+08:00 and status AVAILABLE 2024-03-13 15:55:39.340 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:55:39.340238783+08:00 2024-03-13 15:55:39.341 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:55:39.340+08:00 2024-03-13 15:55:39.341 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:55:39.341 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:55:39.341029141+08:00 2024-03-13 15:55:39.342 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.342 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:55:39.341+08:00 2024-03-13 15:55:39.342 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:55:39.338+08:00 and status AVAILABLE 2024-03-13 15:55:39.343 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:55:39.343175301+08:00 2024-03-13 15:55:39.343 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:55:39.339+08:00 and status AVAILABLE 2024-03-13 15:55:39.344 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:55:39.341+08:00 and status AVAILABLE 2024-03-13 15:55:39.344 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:55:39.343+08:00 2024-03-13 15:55:39.344 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:55:39.340+08:00 and status AVAILABLE 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:55:39.346 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:55:39.346 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:55:39.343+08:00 and status AVAILABLE 2024-03-13 15:55:39.346 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:55:39.347 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:55:39.348 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:55:39.348 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:55:39.349 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:55:39.349 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:55:39.349 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:55:39.350 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:55:39.350145324+08:00 2024-03-13 15:55:39.350 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.351 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:55:39.350+08:00 2024-03-13 15:55:39.351 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:55:39.351 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:55:39.351 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:55:39.351 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:55:39.352 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:55:39.350+08:00 and status AVAILABLE 2024-03-13 15:55:39.353 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:55:39.353 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:55:39.353 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:55:39.357 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:55:39.359 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.361 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.365 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:55:39.365 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:55:39.365 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:55:39.367 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:55:39.367 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:55:39.367 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:55:39.368 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.370 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:39.370 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:39.371 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:39.371 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:55:39.371 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:55:39.371 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:55:39.371 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:39.372 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:39.373 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:39.376 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:39.377 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:39.383 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.386 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:39.387 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:39.387 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:39.388 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.388 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:39.389 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.389 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:39.390 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:39.393 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.402 WARN 1433467 --- [pool-Check host task for clockdiff6,c2b88db545634026,147b8b80b94f] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.402 ERROR 1433467 --- [pool-Check host task for clockdiff6,c2b88db545634026,147b8b80b94f] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:55:39.403 WARN 1433467 --- [pool-Check host task for clockdiff6,c2b88db545634026,147b8b80b94f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.402 WARN 1433467 --- [pool-Check host task for clockdiff2,c2b88db545634026,b96c463cc429] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.403 ERROR 1433467 --- [pool-Check host task for clockdiff2,c2b88db545634026,b96c463cc429] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:55:39.403 WARN 1433467 --- [pool-Check host task for clockdiff2,c2b88db545634026,b96c463cc429] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.403 WARN 1433467 --- [pool-Check host task for clockdiff29,c2b88db545634026,2d127af644d8] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.403 ERROR 1433467 --- [pool-Check host task for clockdiff29,c2b88db545634026,2d127af644d8] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:55:39.403 WARN 1433467 --- [pool-Check host task for clockdiff29,c2b88db545634026,2d127af644d8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.420 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:55:39.421 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:55:39.424 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:55:39.428 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.430 WARN 1433467 --- [pool-Check host task for clockdiff24,c2b88db545634026,6e6a70bc9526] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.430 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:55:39.430 ERROR 1433467 --- [pool-Check host task for clockdiff24,c2b88db545634026,6e6a70bc9526] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:55:39.430 WARN 1433467 --- [pool-Check host task for clockdiff24,c2b88db545634026,6e6a70bc9526] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.430 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:55:39.433 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:55:39.434 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:55:39.438 WARN 1433467 --- [pool-Check host task for clockdiff8,c2b88db545634026,c485b623ec2c] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.438 ERROR 1433467 --- [pool-Check host task for clockdiff8,c2b88db545634026,c485b623ec2c] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:55:39.439 WARN 1433467 --- [pool-Check host task for clockdiff8,c2b88db545634026,c485b623ec2c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.439 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:55:39.449 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.449 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.450 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.451 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.454 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.470 WARN 1433467 --- [pool-Check host task for clockdiff1,c2b88db545634026,544fe2e944c0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.470 ERROR 1433467 --- [pool-Check host task for clockdiff1,c2b88db545634026,544fe2e944c0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:55:39.470 WARN 1433467 --- [pool-Check host task for clockdiff1,c2b88db545634026,544fe2e944c0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.484 WARN 1433467 --- [pool-Check host task for clockdiff3,c2b88db545634026,d1bb7f70dd99] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.484 ERROR 1433467 --- [pool-Check host task for clockdiff3,c2b88db545634026,d1bb7f70dd99] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:55:39.484 WARN 1433467 --- [pool-Check host task for clockdiff3,c2b88db545634026,d1bb7f70dd99] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.492 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:55:39.493 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:55:39.497 WARN 1433467 --- [pool-scanner-1609796195-1,7749f97dd8d6471a,42b2645bbd5e] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:39.499 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:55:39.499 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:55:39.499 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:55:39.506 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:55:39.506 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:55:39.507 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:55:39.508 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:55:39.509 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:55:39.510 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:55:39.512 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.513 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.514 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.514 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.514 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.514 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:55:39.515 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:55:39.518 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.518 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.518 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.524 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.525 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.527 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.533 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.533 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.533 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:55:39.533 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.533 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.534 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:39.535 WARN 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:55:39.535 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.535 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.535 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:55:39.536 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:55:39.537 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:39.540 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:55:39.541 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:55:39.543 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.543 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:39.553 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:55:39.554 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:55:39.554 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.554 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.557 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:55:39.558 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.561 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:55:39.561 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:55:39.561 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:55:39.563 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.563 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.563 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.563 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.564 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:39.564 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:55:39.565 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:55:39.567 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:55:39.568 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:55:39.570 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:39.571 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:39.572 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:55:39.572 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.573 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:55:39.574 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:55:39.578 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:55:39.579 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:55:39.581 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:55:39.584 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.584 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:55:39.604 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:55:39.610 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:55:39.618 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:55:39.623 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:55:39.665 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:55:39.673 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.673 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.673 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.673 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.677 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:55:39.682 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:55:39.682 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710315940072000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:55:39.723 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.724 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.724 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.724 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.724 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:55:39.749 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.749 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.750 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.750 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.751 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:39.755 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.755 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.755 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:55:39.760 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:39.772 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.772 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:39.789 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.789 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:39.804 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.804 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:39.817 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.817 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:39.825 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.825 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:39.833 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.843 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.843 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.843 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.843 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.844 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:39.848 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:55:39.852 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:55:39.855 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:39.857 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:39.877 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:55:39.899 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805159,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:39.911 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:55:39.925 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.926 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.926 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.926 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.927 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:55:39.938 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:55:39.938 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:55:40.068 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,08160f5ea5874bc7,21a2e21917d9] c.o.o.s.c.trace.RequestTracingAspect : API: [POST /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=obcluster&version=2&ObClusterId=1, client=10.10.10.148, traceId=08160f5ea5874bc7, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.updateRootServiceInfo(String,Long,String,Long,Integer,String), args=obcluster,1,2,{"ObRegion":"obcluster","ObCluster":"obcluster","ObRegionId":1,"ObClusterId":1,"Type":"PRIMARY","timestamp":1710316540066489,"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"ReadonlyRsList":[]},] 2024-03-13 15:55:40.083 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,08160f5ea5874bc7,21a2e21917d9] c.o.ocp.configurl.ConfigUrlService : updateRootServer, clusterName=obcluster, clusterId=1, affectRows=1 2024-03-13 15:55:40.084 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,08160f5ea5874bc7,21a2e21917d9] c.o.o.s.c.trace.RequestTracingAspect : API OK: [POST /services client=10.10.10.148, traceId=08160f5ea5874bc7, duration=16 ms] 2024-03-13 15:55:40.137 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:40.138 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:41.899 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805182,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:43.138 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:43.140 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:43.900 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805205,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:45.900 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805227,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:46.105 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805231,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:46.140 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:46.141 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:46.748 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:46.747565+08:00, lastProcessTime=2024-03-13T15:55:36.747607+08:00 2024-03-13 15:55:46.840 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:46.747565+08:00 2024-03-13 15:55:46.948 INFO 1433467 --- [pool-ocp-schedules-8,fef14601c0114840,4d5108179d2d] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:47.901 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805251,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:47.908 INFO 1433467 --- [pool-clean-time-series-data15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:55:47.924 INFO 1433467 --- [pool-print-time-series-interval15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316547: {1710316440=1394, 1710316455=1394, 1710316470=1394, 1710316485=1394, 1710316500=1394, 1710316515=1394, 1710316530=1394, 1710316545=60}, tsDataMapSize: 1122624, readyDataCacheSize: 8288 2024-03-13 15:55:47.949 INFO 1433467 --- [pool-clean-time-series-data60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:55:47.949 INFO 1433467 --- [pool-print-time-series-interval60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316547: {1710316380=34, 1710316440=32, 1710316500=6}, tsDataMapSize: 17768, readyDataCacheSize: 18488 2024-03-13 15:55:49.141 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:49.143 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:49.275 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805268,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:49.349 WARN 1433467 --- [pool-scanner-1609796811-1,167629ef5db04233,c447da5acebf] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:49.417 INFO 1433467 --- [alarm-detect-4,c5ee5d35b14b4195,af0c70a1884b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:55:49.419 INFO 1433467 --- [alarm-detect-4,c5ee5d35b14b4195,af0c70a1884b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316515 ~ 1710316539 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316530=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316515 ~ 1710316539 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316530=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316515 ~ 1710316539 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316530=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316515 ~ 1710316539 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316530=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316515 ~ 1710316539 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316530=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.a.core.detect.AlarmRuleDetector : detectByMetric done, alarmType=host_ntp_service_not_exist, needSendingCount=1, detectAlarmsCount=1, matchedTargets=[alarm_template_id=3:host=10.10.10.148-2, alarm_template_id=3:host=10.10.10.146-2, alarm_template_id=3:host=10.10.10.145-2, alarm_template_id=3:host=10.10.10.144-2, alarm_template_id=3:host=10.10.10.143-2] 2024-03-13 15:55:49.901 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805275,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:51.902 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805298,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:52.143 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:52.144 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:53.176 INFO 1433467 --- [pool-ocp-schedules-8,761291c56aa14358,243a7aaa6071] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:55:53.184 INFO 1433467 --- [pool-ocp-schedules-8,761291c56aa14358,243a7aaa6071] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:53.184 INFO 1433467 --- [pool-ocp-schedules-8,761291c56aa14358,243a7aaa6071] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:53.902 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805321,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:55.144 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:55.145 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:55.713 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2024-03-13 15:55:55.713 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=609459010050985984 2024-03-13 15:55:55.717 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2024-03-13 15:55:55.717 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624101396131614721 2024-03-13 15:55:55.721 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2024-03-13 15:55:55.721 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624425259050733570 2024-03-13 15:55:55.902 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805344,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:56.105 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805348,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:56.274 INFO 1433467 --- [pool-Check obproxy12,7ef44117d4f14915,330dae270639] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:55:56.276 INFO 1433467 --- [pool-Check obproxy13,7ef44117d4f14915,da50881edfbe] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:55:56.276 INFO 1433467 --- [pool-Check obproxy5,7ef44117d4f14915,62c8313d50c4] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:55:56.276 INFO 1433467 --- [pool-Check obproxy8,7ef44117d4f14915,7dd6bdf06d70] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:55:56.747 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:56.746743+08:00, lastProcessTime=2024-03-13T15:55:46.747565+08:00 2024-03-13 15:55:56.869 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:56.746743+08:00 2024-03-13 15:55:56.955 INFO 1433467 --- [pool-ocp-schedules-6,696dc1cdbbce4d5b,a6005d48e5ab] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:57.903 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805368,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:58.145 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:58.146 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:59.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805383,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:59.282 INFO 1433467 --- [alarm-detect-2,7c76b2805c9a477d,d5ac744cb4c4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:55:59.283 INFO 1433467 --- [alarm-detect-16,d96dbbb705b14f43,da0cc72a338f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:55:59.284 INFO 1433467 --- [alarm-detect-20,473880d2e31a47ad,541c2733932d] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:55:59.287 INFO 1433467 --- [alarm-detect-2,7c76b2805c9a477d,d5ac744cb4c4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:55:59.288 INFO 1433467 --- [alarm-detect-16,d96dbbb705b14f43,da0cc72a338f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:55:59.289 INFO 1433467 --- [alarm-detect-20,473880d2e31a47ad,541c2733932d] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:55:59.333 WARN 1433467 --- [pool-scanner-1609797397-1,09650d6c7e6940b1,1cff461a346e] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:59.430 INFO 1433467 --- [alarm-detect-17,70107dc2a1b145d0,dec3aa7e245e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316530 ~ 1710316549 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316545=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:55:59.451 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805383,,] c.o.o.a.c.t.DistributedAlarmDetectTask : Refresh detector factory. 2024-03-13 15:55:59.903 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805391,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:00.566 INFO 1433467 --- [pool-prometheus-minute-schedule-1,7daf0c38589b4c3c,6432c89e7dd1] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1710316620, interval=60 2024-03-13 15:56:00.567 INFO 1433467 --- [pool-prometheus-cache-monitor1,e81a42b7262d4922,0a4aca0ebce0] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":12075,"minuteWriteQueue":0,"minuteReadCacheSize":12328,"secondWriteCacheSize":3328,"seriesIdKeySize":67930,"secondWriteQueue":0,"minuteWriteCacheSize":12393} 2024-03-13 15:56:00.567 INFO 1433467 --- [pool-prometheus-cache-monitor1,e81a42b7262d4922,0a4aca0ebce0] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{go_memory_classes_heap_stacks_bytes=15, ob_sysstat_40012=19, ob_server_resource_memory_assigned_bytes=13, ob_sysstat_40011=19, ob_sysstat_40010=19, ob_unit_num=13, ob_database_disk_required_size=35, go_memstats_mspan_sys_bytes=15, go_memstats_stack_sys_bytes=15, ob_tenant_request_queue_total_size=18, jvm_gc_memory_allocated_bytes_total=3, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=18, ob_sysstat_40019=19, observer_fd_count=13, ob_sysstat_40018=19, processor_process_metrics_total=21, jvm_gc_max_data_size_bytes=3, go_memstats_frees_total=15, sql_audit_input_collect_data_jumped_total=8, host_available=64, mysql_output_write_sql_failed_count=17, ob_database_disk_data_size=34, ob_system_event_detail_total_timeouts=763, ob_tenant_memstore_freeze_cnt=14, ob_latch_sleep_total=312, ob_tenant_unit_min_memory=25, ob_sysstat_60087=19, ob_memstore_total_bytes=20, metric_cardinality_count=12, process_resident_memory_bytes=15, jvm_threads_states_threads=8, ob_cache_size_bytes=34, ob_tenant_unit_max_memory=25, input_collect_metrics_total=26, ob_sysstat_60083=19, ob_sysstat_50039=19, ob_sysstat_50038=19, node_disk_io_time_weighted_seconds_total=33, process_open_fds=15, ob_sysstat_50037=19, process_memory_rss_bytes=33, ob_server_resource_cpu_assigned_percent=13, ob_sysstat_50030=19, go_memory_classes_profiling_buckets_bytes=15, go_memstats_heap_inuse_bytes=15, ob_sysstat_40118=19, ob_sysstat_40117=19, ob_sysstat_40116=19, node_memory_memfree_bytes=26, ob_sysstat_40005=19, process_exists=43, ob_sysstat_40004=19, ob_sysstat_40003=19, ob_sysstat_40002=19, ob_sysstat_40001=19, go_gc_duration_seconds=20, go_memory_classes_metadata_mspan_inuse_bytes=15, ob_server_resource_disk_bytes=13, ob_sysstat_40000=19, odp_current_session=26, process_thread_count=33, ob_sysstat_50040=19, ob_sysstat_40009=19, ob_sysstat_40008=19, ob_disk_free_bytes=13, node_filesystem_avail_bytes=72, ob_sysstat_40007=19, ob_sysstat_40006=19, ob_backup_storage_capacity_bytes=650, ob_unit_config_max_iops=21, ob_plan_cache_memory_bytes=18, node_network_transmit_errs_total=28, go_gc_duration_seconds_count=15, monagent_oceanbase_plan_explain_size=4, ob_sysstat_50011=19, ob_sysstat_50010=19, node_disk_read_bytes_total=33, ob_compaction_is_suspended=19, mysql_output_write_sql_milliseconds_summary_count=17, go_memstats_lookups_total=15, mysql_output_write_sql_count=17, process_cpu_percent=33, ob_sysstat_60057=19, host_agent_process_status=18, ob_server_resource_cpu=13, ob_tenant_server_data_index_row_count=20, go_memstats_alloc_bytes=15, go_sched_gomaxprocs_threads=15, node_cpu_seconds_total_native=38, ob_sysstat_140002=19, ob_sysstat_140005=19, ob_sysstat_140006=19, ob_sysstat_140003=19, ob_tenant_resource_min_memory=18, ob_memstore_freeze_times=20, ob_sysstat_50022=19, jvm_gc_pause_seconds_count=8, ob_sysstat_50021=19, jvm_gc_overhead_percent=3, go_memory_classes_heap_unused_bytes=15, ob_cluster_status_check=13, jvm_memory_committed_bytes=10, node_net_bandwidth_bps=29, monagent_oceanbase_db_pool_idle=4, ob_tenant_server_data_index_data_size=20, go_gc_heap_allocs_by_size_bytes_total_count=15, go_memstats_heap_idle_bytes=15, go_gc_cycles_forced_gc_cycles_total=15, go_gc_pauses_seconds_total_bucket=26, ob_server_resource_memory_bytes=13, ob_tenant_assigned_mem_assigned=18, ob_tenant_memstore_minor_freeze_times=14, ob_tenant_server_required_size=22, ob_sysstat_60031=19, process_fd_count=33, ob_tenant_unit_max_cpu=25, ob_latch_wait_seconds_total=312, go_memory_classes_metadata_mcache_free_bytes=15, node_memory_buffers_bytes=26, ob_sysstat_50009=19, go_memstats_heap_released_bytes=15, node_network_receive_bytes_total=28, ob_sysstat_50005=19, ob_sysstat_50008=19, ob_sysstat_50001=19, ob_sysstat_50004=19, exposer_request_latencies_count=6, ob_sysstat_50000=19, ob_session_active_num=18, mysql_output_context_timeout_discard_metrics_count=9, go_memory_classes_metadata_mcache_inuse_bytes=15, node_disk_read_time_seconds_total=33, ob_sysstat_60021=19, ob_sysstat_60020=19, ob_sysstat_60024=19, ob_compaction_frozen_version=19, ob_sysstat_60023=19, metric_count=15, ob_sysstat_60022=19, obproxysys_connectable=1556, ob_tenant_assigned_cpu_assigned=18, mysql_output_write_sql_retry_count=17, ob_tenant_unit_min_cpu=25, ob_unit_config_max_cpu=21, http_request_milliseconds_summary_count=75, monagent_oceanbase_audit_timeinfo_cache_size=4, http_request_milliseconds_summary=78, ob_latch_immediate_miss_total=312, ob_latch_spin_get_total=312, ob_log_level_count=17, monagent_pipeline_report_metrics_total=16, exposer_transferred_bytes_total=6, ob_tenant_context_memory_count=841, node_disk_reads_completed_total=33, ob_sysstat_60019=19, ob_sysstat_60003=19, ob_sysstat_60002=19, ob_sysstat_60001=19, ob_sysstat_60000=19, ob_latch_miss_total=312, go_sched_latencies_seconds_count=15, ocp_meta_db_health=5, ob_disk_total_bytes=13, go_memstats_heap_objects=15, go_memstats_mcache_inuse_bytes=15, ob_sysstat_60005=19, ob_sysstat_60004=19, go_memory_classes_other_bytes=15, ob_memstore_active_bytes=20, go_memstats_alloc_bytes_total=15, ob_tenant_server_partition_replica_count=21, process_count=34, ob_system_event_detail_total_waits=763, odp_used_connections=25, backup_storage_capacity_used_percent=8, go_gc_heap_frees_objects_total=15, go_memory_classes_metadata_mspan_free_bytes=15, ob_disk_log_used_bytes=13, ob_compaction_is_error=19, go_gc_limiter_last_enabled_gc_cycle=15, monagent_pipeline_execute_total=24, node_filesystem_files=86, process_start_time_seconds=15, go_memstats_heap_alloc_bytes=15, ob_tenant_resource_max_cpu=18, ntp_service_available=26, go_memstats_mspan_inuse_bytes=15, mysql_output_write_sql_milliseconds_summary=20, go_gc_heap_frees_by_size_bytes_total_count=15, monagent_pipeline_execute_seconds_total=24, ob_sysstat_80001=19, go_memstats_sys_bytes=15, go_memory_classes_heap_free_bytes=15, system_load_average_1m=3, monagent_oceanbase_audit_stat_cache_size=4, ob_tenant_memory_used_bytes=22, obproxy_server_max_connections=7, ob_sysstat_80002=19, go_memstats_gc_cpu_fraction=15, ob_sysstat_80003=19, monagent_oceanbase_audit_cache_size=4, ob_tenant_context_memory_used_bytes=841, ob_sysstat_80007=19, node_disk_writes_completed_total=33, ob_sysstat_80008=19, ob_sysstat_80009=19, go_sched_goroutines_goroutines=15, go_memory_classes_metadata_other_bytes=15, go_gc_cycles_total_gc_cycles_total=15, system_cpu_usage=3, process_cpu_usage=3, node_disk_written_bytes_total=33, monagent_oceanbase_ash_cache_size=4, go_goroutines=15, ob_sysstat_80023=19, exposer_request_latencies_sum=6, ob_unit_config_min_cpu=21, process_virtual_memory_max_bytes=15, go_gc_heap_frees_bytes_total=15, ob_tenant500_memory_hold_bytes=13, exposer_scrapes_total=6, go_gc_heap_allocs_by_size_bytes_total_bucket=27, ob_sysstat_80025=19, ob_sysstat_80026=19, oceanbase_connectivity=15, node_memory_sreclaimable_bytes=26, go_sched_latencies_seconds_sum=15, tcp_retrans=25, exporter_export_metrics_bytes_total=26, node_filesystem_size_bytes=72, http_server_requests_seconds_count=203, ob_compaction_frozen_time=19, monagent_input_sql_duration_seconds_sum=55, log_stream_degraded_count=18, ob_plan_cache_hit_total=18, node_filesystem_readonly=72, ob_sysstat_80040=19, ob_sysstat_80041=19, io_await=52, go_gc_heap_allocs_objects_total=15, go_memstats_mallocs_total=15, ob_unit_config_max_memory_bytes=21, go_gc_stack_starting_size_bytes=15, io_util=52, ob_sysstat_200002=19, ob_sysstat_200001=19, ob_tenant_disk_v4_data_size=18, go_sched_latencies_seconds_bucket=26, ob_tenant_assigned_cpu_total=18, ob_tenant_server_available_disk_free=19, node_disk_write_time_seconds_total=33, cpu_count=25, output_write_metrics_total=9, ob_cluster_sync=15, go_gc_heap_objects_objects=15, odp_transaction_total=16, ob_system_event_detail_time_waited_us=763, process_max_fds=15, http_status_count=20, ob_sysstat_80057=19, ocp_alarm_detect_duration_ms_count=115, go_gc_heap_allocs_by_size_bytes_total_sum=15, go_memstats_mcache_sys_bytes=15, ob_tenant_disk_v4_log_size=18, go_cgo_go_to_c_calls_calls_total=15, ocp_alarm_detect_duration_ms_sum=115, jvm_threads_live_threads=3, node_memory_cached_bytes=26, ob_table_num=16, ob_sysstat_10000=19, ob_sysstat_10006=19, ob_sysstat_10005=19, ob_sysstat_10002=19, ob_sysstat_10001=19, go_memstats_last_gc_time_seconds=15, ob_sysstat_10004=19, ob_sysstat_10003=19, monitor_exporter_avaliable=62, log_tailer_count=15, ob_server_num=25, ob_session_all_num=18, ob_system_event_time_waited=24, ob_tenant_resource_min_cpu=18, monagent_plugin_execute_seconds_total=39, go_gc_cycles_automatic_gc_cycles_total=15, go_gc_heap_allocs_bytes_total=15, go_memstats_gc_sys_bytes=15, ob_latch_get_total=312, go_info=16, log_tailer_processed_file_count=21, node_load15=24, ob_disk_log_assigned_bytes=13, go_memory_classes_heap_released_bytes=15, jvm_memory_max_bytes=10, ob_unit_config_min_iops=21, ob_sysstat_30080=19, ob_sysstat_30082=19, ob_sysstat_30081=19, ob_sysstat_30084=19, ob_sysstat_30083=19, ob_sysstat_30086=19, ob_sysstat_30085=19, ob_tenant_log_disk_total_bytes=18, ob_memstore_freeze_trigger_bytes=20, ob_waitevent_wait_seconds_total=18, ob_server_stopped_duration_seconds=15, go_memstats_next_gc_bytes=15, node_filesystem_files_free=86, ob_memtable_max_snapshot_duration_seconds=18, monagent_oceanbase_audit_unique_key_cache_size=4, process_virtual_memory_bytes=15, ob_sysstat_20002=19, net_connectivity=15, ob_sysstat_20001=19, monagent_input_sql_duration_seconds_bucket=65, ob_latch_immediate_get_total=312, node_cpu_seconds_total=32, ob_sysstat_20000=19, ob_index_error_num=13, ob_tenant_server_available_disk_total=19, ob_tenant_memory_hold_bytes=22, ob_sysstat_30066=19, go_memstats_buck_hash_sys_bytes=15, mysql_output_write_sql_milliseconds_summary_sum=17, ob_tenant_disk_total_size=18, odp_sql_request_total=53, mysql_output_write_task_size=17, exporter_export_metrics_total=26, ob_disk_log_total_bytes=13, system_cpu_count=3, ob_sysstat_130004=19, process_uptime_seconds=33, node_ntp_offset_seconds=25, go_memstats_heap_sys_bytes=15, ob_sysstat_140012=19, ob_sysstat_140013=19, jvm_gc_pause_seconds_max=8, ob_tenant_request_queue_large_size=18, ob_sysstat_130000=19, go_memstats_other_sys_bytes=15, ob_sysstat_130001=19, jvm_memory_used_bytes=10, ob_sysstat_130002=19, go_threads=15, http_request_milliseconds_summary_sum=75, http_server_requests_seconds_sum=203, ob_tenant_assigned_mem_total=18, node_memory_memtotal_bytes=26, node_network_transmit_drop_total=28, go_gc_pauses_seconds_total_count=15, exposer_request_latencies=9, ob_waitevent_wait_total=18, go_gc_heap_goal_bytes=15, ob_tenant_log_disk_used_bytes=18, jvm_gc_memory_promoted_bytes_total=3, odp_sql_cost_total=51, ob_sysstat_30011=19, ob_sysstat_30010=19, ob_sysstat_30013=19, ob_sysstat_30012=19, ob_tenant500_memory_used_bytes=13, node_network_transmit_bytes_total=28, node_network_receive_drop_total=28, go_gc_heap_tiny_allocs_objects_total=15, go_gc_duration_seconds_sum=15, ob_server_resource_memory_assigned_percent=13, jvm_gc_pause_seconds_sum=8, go_memory_classes_heap_objects_bytes=15, node_network_receive_errs_total=28, observer_ulimit_max_fd_count=13, ob_plan_cache_access_total=18, ob_tenant_server_row_count=22, monagent_oceanbase_plan_cache_size=4, ob_tenant_server_data_size=22, monagent_oceanbase_audit_sample_cache_size=4, go_gc_heap_frees_by_size_bytes_total_sum=15, ob_sysstat_40030=19, node_load1=24, process_cpu_seconds_total=15, ob_sysstat_120001=19, go_memory_classes_total_bytes=15, ob_sysstat_120002=19, ob_sysstat_120000=19, ob_tenant_resource_max_memory=18, ob_sysstat_120009=19, go_memstats_stack_inuse_bytes=15, ob_sysstat_120008=19, node_load5=24, ob_sysstat_120005=19, go_memory_classes_os_stacks_bytes=15, ob_sysstat_120006=19, jvm_threads_peak_threads=3, monagent_input_sql_duration_seconds_count=55, ob_tenant_context_memory_hold_bytes=841, obproxy_memory_limit_bytes=7, ob_compaction_last_version=19, ob_sysstat_30000=19, ob_server_resource_cpu_assigned=13, ob_sysstat_30002=19, ob_sysstat_30001=19, monagent_oceanbase_plan_timeinfo_cache_size=4, monagent_plugin_execute_total=39, ob_sysstat_30006=19, ob_sysstat_30005=19, ob_sysstat_30008=19, ob_sysstat_30007=19, ob_sysstat_30009=19, go_gc_heap_frees_by_size_bytes_total_bucket=27, obproxy_client_max_connections=7, ob_compaction_current_timestamp=19, ob_clog_max_sync_delay_seconds=20, go_gc_pauses_seconds_total_sum=15, ob_compaction_start_time=19} 2024-03-13 15:56:00.573 INFO 1433467 --- [pool-prometheus-minute-schedule-1,7daf0c38589b4c3c,6432c89e7dd1] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=18, collectTaskCount=18 2024-03-13 15:56:00.573 INFO 1433467 --- [pool-prometheus-minute-schedule-1,7daf0c38589b4c3c,6432c89e7dd1] c.o.ocp.monitor.OcpMonitorManager : Schedule interval metric too slow, from=1710316560, count=18, elapsed=1708606244013 2024-03-13 15:56:00.593 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:00.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:00 CST 2024 2024-03-13 15:56:00.593 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=38, name=Detect alarms on schedule, templateName=BackupScheduleAlarmTemplateFactory, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:55:00.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:00.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Detect alarms, nodeCount=1, taskCount=1 2024-03-13 15:56:00.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:00.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Detect alarms, taskDefinitionId=38, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:00.602 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:00.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:00 CST 2024 2024-03-13 15:56:00.602 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999975, name=OasSqlCuring, templateName=OasSqlCuringSchedule, scheduleRule=0 */2 * * * *, lastRunTime=2024-03-13 15:54:00.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:00.624 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : Split list is empty, name=OasSqlCuringSchedule, splitKey=cluster_ids, context=Argument{parallelIdx=-1, stringMap={}, listMap={cluster_ids=[]}} 2024-03-13 15:56:00.624 WARN 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule cron task failed. java.lang.RuntimeException: Illegal split list, the list is empty, splitKey=cluster_ids at com.oceanbase.ocp.core.metadb.repo.task.model.TaskInstanceEntity.generateSubtasks(TaskInstanceEntity.java:150) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.buildTaskInstance(ReadyTaskWorker.java:270) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.submitTask(ReadyTaskWorker.java:210) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.doScheduleTask(ReadyTaskWorker.java:197) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.checkAndScheduleCronTask(ReadyTaskWorker.java:173) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.lambda$loopScheduleTask$1(ReadyTaskWorker.java:133) at org.springframework.transaction.support.TransactionOperations.lambda$executeWithoutResult$0(TransactionOperations.java:68) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) at org.springframework.transaction.support.TransactionOperations.executeWithoutResult(TransactionOperations.java:67) at com.oceanbase.ocp.core.task.engine.coordinator.worker.AbstractTaskWorker.transactionWithoutResult(AbstractTaskWorker.java:62) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.loopScheduleTask(ReadyTaskWorker.java:132) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.work(ReadyTaskWorker.java:101) at com.oceanbase.ocp.core.task.engine.coordinator.TaskCoordinator.lockAndExecute(TaskCoordinator.java:94) at com.oceanbase.ocp.core.task.engine.coordinator.TaskCoordinator.lambda$startup$0(TaskCoordinator.java:65) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:00.629 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:01.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:00 CST 2024 2024-03-13 15:56:00.629 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999981, name=Collect all tenants major compactions, templateName=CollectTenantCompactionSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:55:01.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:00.638 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all tenants major compactions, nodeCount=1, taskCount=1 2024-03-13 15:56:00.638 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:00.638 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all tenants major compactions, taskDefinitionId=999981, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:00.644 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:01.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:00 CST 2024 2024-03-13 15:56:00.644 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999982, name=Collect SlowSql to SqlAggregateData for alarm, templateName=CollectSlowSqlSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:55:01.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:00.926 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect SlowSql to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:56:00.926 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:00.926 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect SlowSql to SqlAggregateData for alarm, taskDefinitionId=999982, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:00.932 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:01.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:00 CST 2024 2024-03-13 15:56:00.932 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999983, name=Collect trans stat to SqlAggregateData for alarm, templateName=CollectTransSatatSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:55:01.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:01.146 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:01.147 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:01.192 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect trans stat to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:56:01.192 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:01.192 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect trans stat to SqlAggregateData for alarm, taskDefinitionId=999983, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:01.197 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:01.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:01 CST 2024 2024-03-13 15:56:01.197 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999984, name=Get agent alarms, templateName=GetAgentAlarmsSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:55:01.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:01.200 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Get agent alarms, nodeCount=1, taskCount=1 2024-03-13 15:56:01.200 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:01.200 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Get agent alarms, taskDefinitionId=999984, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:01.207 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:01.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:01 CST 2024 2024-03-13 15:56:01.207 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999992, name=Sync root service list, templateName=SyncRsListSchedule, scheduleRule=0 */2 * * * ?, lastRunTime=2024-03-13 15:54:01.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:01.210 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Sync root service list, nodeCount=1, taskCount=2 2024-03-13 15:56:01.210 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:01.210 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:01.210 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Sync root service list, taskDefinitionId=999992, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:01.217 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:31.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:01 CST 2024 2024-03-13 15:56:01.217 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:55:31.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:01.218 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:56:01.219 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:01.219 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:01.496 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016747, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.512 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016748, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.521 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016747, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014397, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.493+08:00, sub_task_instance_name=Detect alarms of backup, sub_task_instance_id=10016747}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.524 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016749, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.531 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,d2294683914b] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016748, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014398, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.509+08:00, sub_task_instance_name=Collect all tenants major compactions, sub_task_instance_id=10016748}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.531 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,d2294683914b] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2024-03-13 15:56:01.543 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016750, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.544 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,6d25376f25c1] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016749, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014399, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.522+08:00, sub_task_instance_name=Collect SlowSql to SqlAggregateData for alarm, sub_task_instance_id=10016749}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.546 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.549 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:56:01.550 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:56:01.550 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:56:01.550 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:56:01.551 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.554 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.559 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.560 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.560 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:56:01.560 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.563 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.561 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016751, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.562 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.564 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,19212c62189b] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016750, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014400, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.539+08:00, sub_task_instance_name=Collect trans stat to SqlAggregateData for alarm, sub_task_instance_id=10016750}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.564 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:56:01.564 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.564 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.566 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.566 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.567 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.567 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:56:01.568 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266400687471944, broadcast scn:1710266400687471944, saved frozen scn:1710266400687471944 2024-03-13 15:56:01.568 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402011840869, broadcast scn:1710266402011840869, saved frozen scn:1710266402011840869 2024-03-13 15:56:01.568 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:56:01.568 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:56:01.569 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402269282986, broadcast scn:1710266402269282986, saved frozen scn:1710266402269282986 2024-03-13 15:56:01.569 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:56:01.570 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266403004770377, broadcast scn:1710266403004770377, saved frozen scn:1710266403004770377 2024-03-13 15:56:01.570 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:56:01.570 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,d2294683914b] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2024-03-13 15:56:01.574 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,d2294683914b] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016748, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.577 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016752, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.579 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,e774edb5c5e2] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016751, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014401, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.557+08:00, sub_task_instance_name=Get all agent alarms, sub_task_instance_id=10016751}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.585 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,19212c62189b] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster1, trans stat result is empty. 2024-03-13 15:56:01.585 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,19212c62189b] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster, trans stat result is empty. 2024-03-13 15:56:01.587 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016753, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.588 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,6d25376f25c1] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for obcluster1, save 1 to SqlAggregateData 2024-03-13 15:56:01.589 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,19212c62189b] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016750, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.590 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,6d25376f25c1] c.o.o.p.s.i.alarm.SlowSqlDataSource : query sqlCount of obCluster obcluster, sql count is 0. 2024-03-13 15:56:01.589 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,6644df4cf228] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016752, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014402, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.575+08:00, sub_task_instance_name=Refresh root service list, sub_task_instance_id=10016752}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:56:01.590 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,6d25376f25c1] c.o.o.p.s.i.alarm.SlowSqlDataSource : query SlowSqlData of obCluster obcluster, result is empty. 2024-03-13 15:56:01.593 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,6d25376f25c1] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016749, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.596 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016754, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2024-03-13 15:56:01.599 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,6644df4cf228] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016752, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=7 2024-03-13 15:56:01.599 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,11075ac7b7fd] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016753, context=Context{parallelIdx=1, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014402, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.585+08:00, sub_task_instance_name=Refresh root service list, sub_task_instance_id=10016753}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=6 2024-03-13 15:56:01.601 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,e774edb5c5e2] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016751, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.602 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:01.602 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:01.602 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:01.603 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:01.603 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:01.603 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:01.603 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:01.604 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:01.604 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:01.605 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:01.605 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:01.607 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:01.607 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,11075ac7b7fd] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016753, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.607 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:01.608 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:01.614 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.615 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,8d2c3ae7cfab] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016754, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014403, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.594+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016754}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.616 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.618 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.620 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.622 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.623 INFO 1433467 --- [pool-Check obproxy15,f47a761474a84dce,d16b37d2323f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:56:01.623 INFO 1433467 --- [pool-Check obproxy10,f47a761474a84dce,57f7c8d91b14] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:56:01.624 INFO 1433467 --- [pool-Check obproxy9,f47a761474a84dce,681ad4e1d7d1] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:56:01.625 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.626 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.626 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,693a7d504e57] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:56:01.639 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.640 INFO 1433467 --- [pool-Check obproxy10,f47a761474a84dce,57f7c8d91b14] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:56:01.642 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.643 INFO 1433467 --- [pool-Check obproxy9,f47a761474a84dce,681ad4e1d7d1] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:56:01.644 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.646 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.648 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:56:01.650 INFO 1433467 --- [pool-Check obproxy15,f47a761474a84dce,d16b37d2323f] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:56:01.660 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,693a7d504e57] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:56:01.668 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,8d2c3ae7cfab] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016754, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.670 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:56:01.687 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.690 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.692 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.695 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:56:01.738 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.741 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.744 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.745 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.748 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:56:01.769 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:56:01.793 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.796 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.810 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.813 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.816 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.819 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.822 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:56:01.825 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:56:01.829 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.832 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.834 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [3600] 2024-03-13 15:56:01.877 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=3600, delayTasks=[] 2024-03-13 15:56:01.889 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.892 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.904 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805414,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:01.913 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.916 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.920 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.923 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.926 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:56:01.930 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:56:01.934 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.937 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.942 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [300] 2024-03-13 15:56:01.974 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=300, delayTasks=[] 2024-03-13 15:56:02.001 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:02.008 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:02.011 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:02.014 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:02.019 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [90] 2024-03-13 15:56:02.059 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=90, timeoutTasks=[] 2024-03-13 15:56:02.077 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:02.081 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:02.085 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:02.088 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:02.092 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [60] 2024-03-13 15:56:02.123 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=60, timeoutTasks=[] 2024-03-13 15:56:02.141 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:02.144 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:02.147 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:02.149 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:02.153 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:56:02.191 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:56:02.201 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:02.204 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:02.207 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:02.209 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:02.213 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:56:02.246 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:56:02.250 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster1 does not enable secondary backup 2024-03-13 15:56:02.253 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster does not enable secondary backup 2024-03-13 15:56:02.259 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackup/], metricValues size:[0]. 2024-03-13 15:56:02.263 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackupocp/], metricValues size:[0]. 2024-03-13 15:56:02.277 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016747, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:02.360 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,2b64fd7cebab4e7c,e8ebfe11c707] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.148, traceId=2b64fd7cebab4e7c, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:56:02.365 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,2b64fd7cebab4e7c,e8ebfe11c707] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=2b64fd7cebab4e7c, duration=6 ms] 2024-03-13 15:56:02.565 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014397, status:SUCCESS 2024-03-13 15:56:02.566 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014397, status:SUCCESS 2024-03-13 15:56:02.577 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014398, status:SUCCESS 2024-03-13 15:56:02.578 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014398, status:SUCCESS 2024-03-13 15:56:02.586 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014399, status:SUCCESS 2024-03-13 15:56:02.586 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014399, status:SUCCESS 2024-03-13 15:56:02.594 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014400, status:SUCCESS 2024-03-13 15:56:02.595 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014400, status:SUCCESS 2024-03-13 15:56:02.603 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014401, status:SUCCESS 2024-03-13 15:56:02.603 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014401, status:SUCCESS 2024-03-13 15:56:02.612 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014402, status:SUCCESS 2024-03-13 15:56:02.613 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014402, status:SUCCESS 2024-03-13 15:56:02.623 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014403, status:SUCCESS 2024-03-13 15:56:02.624 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014403, status:SUCCESS 2024-03-13 15:56:03.904 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805437,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:04.147 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:04.147 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:05.904 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805460,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:06.106 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805463,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:56:06.749 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:56:06.748226+08:00, lastProcessTime=2024-03-13T15:55:56.746743+08:00 2024-03-13 15:56:06.864 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:56:06.748226+08:00 2024-03-13 15:56:06.961 INFO 1433467 --- [pool-ocp-schedules-13,bd73e7d2551c45c2,0cfa9d681651] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:56:07.148 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:07.149 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:07.905 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805484,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:09.248 INFO 1433467 --- [pool-ocp-schedules-20,c0b855f694d54a2c,56b898f4fe9d] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2024-03-13 15:56:09.258 INFO 1433467 --- [pool-ocp-schedules-6,8f0f5d1d129c4e1f,9cf07786e889] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:56:09.259 INFO 1433467 --- [pool-ocp-schedules-12,f1093abb6be84a53,77218090e281] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:56:09.259 INFO 1433467 --- [pool-ocp-schedules-16,9515dc0f2bda42c9,d5dbf7880c98] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:56:09.260 INFO 1433467 --- [pool-ocp-schedules-20,c0b855f694d54a2c,56b898f4fe9d] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2024-03-13 15:56:09.261 INFO 1433467 --- [pool-ocp-schedules-6,8f0f5d1d129c4e1f,9cf07786e889] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:56:09.262 INFO 1433467 --- [pool-ocp-schedules-12,f1093abb6be84a53,77218090e281] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:56:09.263 INFO 1433467 --- [pool-ocp-schedules-16,9515dc0f2bda42c9,d5dbf7880c98] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:56:09.269 INFO 1433467 --- [pool-ocp-schedules-1,dc32853a41db4822,34c3f025b955] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:56:09.269 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:56:09.269 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:56:09.269 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:56:09.270 INFO 1433467 --- [pool-ocp-schedules-11,9b3701f2c3914bc0,5d32e79126d4] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:56:09.270 INFO 1433467 --- [pool-Check host task for clockdiff27,9b3701f2c3914bc0,ccbfc4caa86f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:56:09.270 INFO 1433467 --- [pool-Check host task for clockdiff4,9b3701f2c3914bc0,581a53b72626] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:56:09.270 INFO 1433467 --- [pool-Check host task for clockdiff7,9b3701f2c3914bc0,9a586c3aa8ac] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:56:09.271 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:56:09.270 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:56:09.271 INFO 1433467 --- [pool-Check host task for clockdiff30,9b3701f2c3914bc0,20c359cd66f0] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:56:09.271 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:56:09.271 INFO 1433467 --- [pool-Check host task for clockdiff18,9b3701f2c3914bc0,09d972ffedae] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:56:09.271 INFO 1433467 --- [pool-Check host task for clockdiff28,9b3701f2c3914bc0,47a4314bdd1d] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:56:09.272 INFO 1433467 --- [pool-Check host task for clockdiff21,9b3701f2c3914bc0,a4fb5edbdefe] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:56:09.272 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:56:09.276 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:09.277 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805500,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:56:09.278 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:09.280 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:09.281 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:09.282 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:09.282 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:09.282 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:09.283 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:09.283 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:09.283 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:09.283 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:09.284 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:09.285 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:09.285 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:09.285 INFO 1433467 --- [pool-ocp-schedules-5,88fc9c4da24f4bb5,4d8c7b66ddc1] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:56:09.286 INFO 1433467 --- [pool-ocp-schedules-17,7054d8b52feb4e81,5d0127db3f2b] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:56:09.286 INFO 1433467 --- [pool-ocp-schedules-10,a23690bdba70422c,db352dd6173d] c.o.o.s.task.scheduled.ClusterSchedules : Check host version schedule. 2024-03-13 15:56:09.286 INFO 1433467 --- [pool-ocp-schedules-10,a23690bdba70422c,db352dd6173d] o.o.c.t.e.u.DistributedExclusiveExecutor : Empty item list, taskName=Check host task for host version, skip execute 2024-03-13 15:56:09.326 WARN 1433467 --- [pool-Check host task for clockdiff21,9b3701f2c3914bc0,a4fb5edbdefe] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.327 ERROR 1433467 --- [pool-Check host task for clockdiff21,9b3701f2c3914bc0,a4fb5edbdefe] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:56:09.327 WARN 1433467 --- [pool-Check host task for clockdiff21,9b3701f2c3914bc0,a4fb5edbdefe] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.327 WARN 1433467 --- [pool-Check host task for clockdiff7,9b3701f2c3914bc0,9a586c3aa8ac] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.327 ERROR 1433467 --- [pool-Check host task for clockdiff7,9b3701f2c3914bc0,9a586c3aa8ac] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:56:09.327 WARN 1433467 --- [pool-Check host task for clockdiff7,9b3701f2c3914bc0,9a586c3aa8ac] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.328 INFO 1433467 --- [pool-ocp-schedules-8,c90e4d43b634438d,5217090c5e7c] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:56:09.328 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:56:09.328 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:56:09.328 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:56:09.329 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:56:09.330 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:56:09.330 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:56:09.330 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:56:09.334 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:56:09.335 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:09.336 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:09.336 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:56:09.336488937+08:00 2024-03-13 15:56:09.337 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:56:09.337 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:09.337 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:09.337 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:56:09.336+08:00 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:09.339 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:09.339 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.339 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:09.339 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:56:09.338630900+08:00 2024-03-13 15:56:09.339 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:09.340 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:09.340 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.340 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.340 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:56:09.339876814+08:00 2024-03-13 15:56:09.340 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:56:09.340 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:56:09.339+08:00 2024-03-13 15:56:09.340 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:56:09.336+08:00 and status AVAILABLE 2024-03-13 15:56:09.341 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:09.341 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:56:09.340+08:00 2024-03-13 15:56:09.342 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:56:09.342124677+08:00 2024-03-13 15:56:09.342 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:56:09.342+08:00 2024-03-13 15:56:09.342 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:56:09.340+08:00 and status AVAILABLE 2024-03-13 15:56:09.343 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:56:09.339+08:00 and status AVAILABLE 2024-03-13 15:56:09.343 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:56:09.345 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:56:09.346 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:56:09.345246336+08:00 2024-03-13 15:56:09.346 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:56:09.346 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:56:09.342+08:00 and status AVAILABLE 2024-03-13 15:56:09.347 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:56:09.346+08:00 2024-03-13 15:56:09.348 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:56:09.347959669+08:00 2024-03-13 15:56:09.348 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:56:09.348 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:56:09.348 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:56:09.348+08:00 2024-03-13 15:56:09.350 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:56:09.346+08:00 and status AVAILABLE 2024-03-13 15:56:09.350 INFO 1433467 --- [ForkJoinPool-1386577-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.350 INFO 1433467 --- [ForkJoinPool-1386577-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.350 INFO 1433467 --- [ForkJoinPool-1386577-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.350 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:56:09.350620932+08:00 2024-03-13 15:56:09.351 INFO 1433467 --- [ForkJoinPool-1386577-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:09.351 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:56:09.351 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:56:09.351+08:00 2024-03-13 15:56:09.352 INFO 1433467 --- [ForkJoinPool-1386577-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.352 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:56:09.348+08:00 and status AVAILABLE 2024-03-13 15:56:09.353 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.353 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.353 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.353 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:09.354 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:56:09.354 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:56:09.351+08:00 and status AVAILABLE 2024-03-13 15:56:09.354 INFO 1433467 --- [ForkJoinPool-1386577-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.355 INFO 1433467 --- [ForkJoinPool-1386577-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.355 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:56:09.355 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:09.356 INFO 1433467 --- [ForkJoinPool-1386577-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:09.356 INFO 1433467 --- [ForkJoinPool-1386577-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:56:09.356 INFO 1433467 --- [ForkJoinPool-1386577-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:56:09.357 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:09.359 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:56:09.361 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.361 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:56:09.362 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:09.363 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:09.364 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:09.365 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:09.365 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:09.365 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.366 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:56:09.366 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:56:09.366 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:56:09.366 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:09.366 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.367 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.367 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.367 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:09.368 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:09.368 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:09.368 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:56:09.369 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:56:09.369 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:56:09.369 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:09.369 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:09.371 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:56:09.371 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:56:09.371 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:56:09.373 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.375 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.376 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:56:09.376 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:56:09.376 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:56:09.377 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.378 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:56:09.379 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:56:09.379 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:56:09.383 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.384 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.386 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.387 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.389 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.389 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.413 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:56:09.413 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:56:09.413 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:56:09.414 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:56:09.424 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:56:09.428 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:56:09.433 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:56:09.440 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:56:09.443 WARN 1433467 --- [pool-Check host task for clockdiff27,9b3701f2c3914bc0,ccbfc4caa86f] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.443 WARN 1433467 --- [pool-Check host task for clockdiff4,9b3701f2c3914bc0,581a53b72626] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.443 ERROR 1433467 --- [pool-Check host task for clockdiff27,9b3701f2c3914bc0,ccbfc4caa86f] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:56:09.443 ERROR 1433467 --- [pool-Check host task for clockdiff4,9b3701f2c3914bc0,581a53b72626] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:56:09.443 WARN 1433467 --- [pool-Check host task for clockdiff27,9b3701f2c3914bc0,ccbfc4caa86f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.443 WARN 1433467 --- [pool-Check host task for clockdiff4,9b3701f2c3914bc0,581a53b72626] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.445 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:56:09.446 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:56:09.447 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:56:09.448 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:56:09.455 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:56:09.456 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.457 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:56:09.458 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.462 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.467 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.467 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.468 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.468 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.469 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:56:09.469 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.470 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,a3f4636454e24191,6c66d809965e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.143, traceId=a3f4636454e24191, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:56:09.471 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.473 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,a3f4636454e24191,6c66d809965e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.143, traceId=a3f4636454e24191, duration=4 ms] 2024-03-13 15:56:09.476 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.487 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:56:09.487 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff28,9b3701f2c3914bc0,47a4314bdd1d] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.488 ERROR 1433467 --- [pool-Check host task for clockdiff28,9b3701f2c3914bc0,47a4314bdd1d] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff28,9b3701f2c3914bc0,47a4314bdd1d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff30,9b3701f2c3914bc0,20c359cd66f0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.488 ERROR 1433467 --- [pool-Check host task for clockdiff30,9b3701f2c3914bc0,20c359cd66f0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff30,9b3701f2c3914bc0,20c359cd66f0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff18,9b3701f2c3914bc0,09d972ffedae] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.488 ERROR 1433467 --- [pool-Check host task for clockdiff18,9b3701f2c3914bc0,09d972ffedae] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff18,9b3701f2c3914bc0,09d972ffedae] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.493 WARN 1433467 --- [pool-scanner-1609798002-1,fd4510bf4e7545c8,17e3475f8160] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:56:09.494 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:56:09.495 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:56:09.499 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:56:09.502 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:56:09.503 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:56:09.507 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.512 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:56:09.512 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.513 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.513 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:56:09.514 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:56:09.519 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:56:09.519 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.521 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:56:09.522 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.523 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.523 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.523 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.524 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.524 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:09.525 WARN 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:56:09.525 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.525 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.526 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:56:09.526 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:09.527 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:56:09.528 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:56:09.529 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.530 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.530 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:09.533 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:56:09.534 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:56:09.534 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.536 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:56:09.539 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.541 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:56:09.547 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.547 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.548 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.548 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.548 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:09.548 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:56:09.550 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:56:09.553 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.553 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:09.554 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:09.554 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:56:09.558 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:56:09.563 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:56:09.566 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:56:09.566 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:56:09.568 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:56:09.571 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:56:09.574 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:56:09.575 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:56:09.583 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:56:09.626 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:56:09.634 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:56:09.648 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.648 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.648 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.648 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.649 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:56:09.656 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:56:09.656 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316540069000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:56:09.669 INFO 1433467 --- [alarm-detect-2,f35a7e4e7c494233,2e6b1e2a7c2c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:56:09.672 INFO 1433467 --- [alarm-detect-2,f35a7e4e7c494233,2e6b1e2a7c2c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:56:09.682 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.683 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.683 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.683 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.683 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:56:09.711 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.711 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.711 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.711 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.712 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:09.715 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.715 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.715 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:56:09.718 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:09.724 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.724 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:09.731 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.731 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:09.737 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.737 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:09.744 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.744 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:09.750 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.750 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:09.758 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.776 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.777 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.777 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.777 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.778 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:09.782 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:56:09.787 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:56:09.792 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:09.794 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:09.811 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:56:09.844 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:56:09.863 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.864 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.864 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.864 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.865 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:56:09.877 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:56:09.877 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:56:09.905 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805509,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:10.149 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:10.149 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:10.401 INFO 1433467 --- [pool-async-queue-offer-interval60-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 60, queue size = 6, ruleSize = 6. 2024-03-13 15:56:10.403 INFO 1433467 --- [pool-async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 17, ruleSize = 17. 2024-03-13 15:56:10.858 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:10.858 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:10.867 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:10.867 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:10.872 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:10.872 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:10.877 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:10.877 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:11.087 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.087 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:56:11.087 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:56:11.087 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:56:11.117 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:56:11.117 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:56:11.137 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:56:11.137 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:56:11.170 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:56:11.170 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:56:11.170 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total 2024-03-13 15:56:11.170 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_route 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_server_process_request_time 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_hit 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_insert 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_update 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_response 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_sequence 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_partition 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_table 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_request 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_request 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_response 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_commit 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_delete 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_update 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_select 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_slow 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_others 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_sequence 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_large 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_prepare_send_request_time 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_success 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_select 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_delete 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_commit 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_others 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_insert 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.181 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.181 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.181 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.181 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.181 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.186 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.186 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.186 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.186 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.186 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.192 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.906 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805532,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:12.279 INFO 1433467 --- [pool-metric-archive-second-5,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316500, nodeCount=1000, affectRows=1000 2024-03-13 15:56:12.283 INFO 1433467 --- [pool-metric-archive-second-5,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316500, nodeCount=1000, validCount=1000 2024-03-13 15:56:12.288 INFO 1433467 --- [pool-metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316500, nodeCount=333, affectRows=333 2024-03-13 15:56:12.289 INFO 1433467 --- [pool-metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316500, nodeCount=333, validCount=333 2024-03-13 15:56:12.298 INFO 1433467 --- [pool-metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316500, nodeCount=695, affectRows=695 2024-03-13 15:56:12.306 INFO 1433467 --- [pool-metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316500, nodeCount=695, validCount=695 2024-03-13 15:56:12.309 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316500, nodeCount=993, affectRows=993 2024-03-13 15:56:12.311 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316500, nodeCount=992, validCount=992 2024-03-13 15:56:13.150 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:13.151 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:13.186 INFO 1433467 --- [pool-ocp-schedules-11,a2a644a4427e43cc,b78e37ccddbe] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:56:13.194 INFO 1433467 --- [pool-ocp-schedules-11,a2a644a4427e43cc,b78e37ccddbe] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:56:13.194 INFO 1433467 --- [pool-ocp-schedules-11,a2a644a4427e43cc,b78e37ccddbe] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:56:13.906 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805556,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:14.629 INFO 1433467 --- [pool-metric-archive-async-8,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1710316500, nodeCount=2, affectRows=2 2024-03-13 15:56:14.636 INFO 1433467 --- [pool-metric-archive-async-8,,] c.o.o.m.service.OcpMetricArchiveService : Rollup async second to minute, timestamp=1710316500, nodeCount=2, validCount=2 2024-03-13 15:56:15.103 INFO 1433467 --- [pool-prometheus-second-schedule-1,ad0d3266a49947c8,0d53329e9cb3] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=23, collectTaskCount=23 2024-03-13 15:56:15.907 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805579,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:16.106 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805582,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:56:16.151 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:16.152 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:16.747 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:56:16.746917+08:00, lastProcessTime=2024-03-13T15:56:06.748226+08:00 2024-03-13 15:56:16.829 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:56:16.746917+08:00 2024-03-13 15:56:16.968 INFO 1433467 --- [pool-ocp-schedules-7,c4e5766acfba45cb,9ada81bc760d] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:56:17.907 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805603,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:19.152 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:19.153 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:19.276 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805618,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:56:19.286 INFO 1433467 --- [alarm-detect-7,df8c07d1b81547d8,2240b486dc48] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:56:19.287 INFO 1433467 --- [alarm-detect-25,dda16d2c562044bd,850ae50d031e] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:56:19.290 INFO 1433467 --- [alarm-detect-16,302903dbd7884129,229e6106e6d7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:56:19.291 INFO 1433467 --- [alarm-detect-7,df8c07d1b81547d8,2240b486dc48] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:56:19.291 INFO 1433467 --- [alarm-detect-25,dda16d2c562044bd,850ae50d031e] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:56:19.293 INFO 1433467 --- [alarm-detect-16,302903dbd7884129,229e6106e6d7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:56:19.315 WARN 1433467 --- [pool-scanner-1609799659-1,8e626a88e6924ce3,4b94be5981b1] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:56:19.415 INFO 1433467 --- [alarm-detect-10,accabf20acd242b8,635b08f61642] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316545 ~ 1710316569 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316560=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:56:19.415 INFO 1433467 --- [alarm-detect-10,accabf20acd242b8,635b08f61642] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316545 ~ 1710316569 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316560=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:56:19.415 INFO 1433467 --- [alarm-detect-10,accabf20acd242b8,635b08f61642] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316545 ~ 1710316569 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316560=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:56:19.415 INFO 1433467 --- [alarm-detect-10,accabf20acd242b8,635b08f61642] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316545 ~ 1710316569 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316560=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:56:19.908 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805626,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:20.887 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:20.0, nextRun=2024-03-13 15:56:20.0, currentDate=Wed Mar 13 15:56:20 CST 2024 2024-03-13 15:56:20.888 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999996, name=Collect all cluster compaction info, templateName=CollectClusterCompactionSchedule, scheduleRule=20 * * * * *, lastRunTime=2024-03-13 15:55:20.0, nextRunTime=2024-03-13 15:56:20.0} 2024-03-13 15:56:20.890 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all cluster compaction info, nodeCount=1, taskCount=1 2024-03-13 15:56:20.891 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:20.891 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all cluster compaction info, taskDefinitionId=999996, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:21.783 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016755, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:21.804 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a4cd91395a7c] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016755, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014404, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:21.780+08:00, sub_task_instance_name=Collect compaction for all clusters, sub_task_instance_id=10016755}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:21.806 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a4cd91395a7c] c.o.o.s.t.b.c.AbstractAllClusterTask : Collect compaction for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:56:21.807 INFO 1433467 --- [ForkJoinPool-1386583-worker-1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:56:21.807 INFO 1433467 --- [ForkJoinPool-1386583-worker-2,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:56:21.815 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a4cd91395a7c] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016755, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:21.908 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805649,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:22.031 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014404, status:SUCCESS 2024-03-13 15:56:22.032 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014404, status:SUCCESS 2024-03-13 15:56:22.153 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:22.154 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:22.331 INFO 1433467 --- [pool-ocp-schedules-11,32711f98b3c5444f,7d8d814f0e33] c.o.ocp.core.cache.helper.ObQueryCache : Init ob cache. 2024-03-13 15:56:22.344 INFO 1433467 --- [pool-ocp-schedules-11,32711f98b3c5444f,7d8d814f0e33] c.o.ocp.core.cache.helper.ObQueryCache : Finished init ob cache. 2024-03-13 15:56:22.815 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,570efb35811f4b51,928ce3e6c048] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.142, traceId=570efb35811f4b51, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:56:22.820 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,570efb35811f4b51,928ce3e6c048] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.142, traceId=570efb35811f4b51, duration=6 ms] 2024-03-13 15:56:23.665 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,c67902ce2c204210,2a9a7f1ba96b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.144, traceId=c67902ce2c204210, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:56:23.670 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,c67902ce2c204210,2a9a7f1ba96b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.144, traceId=c67902ce2c204210, duration=6 ms] 2024-03-13 15:56:23.909 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805672,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:25.155 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:25.155 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:25.909 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805695,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:26.107 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805698,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:56:26.341 INFO 1433467 --- [pool-Check obproxy7,12ab4a6ab3e54d10,4f070ed55d6b] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:56:26.342 INFO 1433467 --- [pool-Check obproxy2,12ab4a6ab3e54d10,b50bee226dbe] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:56:26.344 INFO 1433467 --- [pool-Check obproxy3,12ab4a6ab3e54d10,3dc9bdfbe902] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:56:26.345 INFO 1433467 --- [pool-Check obproxy6,12ab4a6ab3e54d10,58c32f765592] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:56:26.742 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:56:26.741760+08:00, lastProcessTime=2024-03-13T15:56:16.746917+08:00 2024-03-13 15:56:26.828 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:56:26.741760+08:00 2024-03-13 15:56:26.974 INFO 1433467 --- [pool-ocp-schedules-19,d7406e5f48044000,35a9729c8ca6] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:56:27.910 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805719,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:28.156 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:28.157 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:29.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805735,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:56:29.331 WARN 1433467 --- [pool-scanner-1609800315-1,a510478317234dc5,b91d2f7f1ae0] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:56:29.417 INFO 1433467 --- [alarm-detect-25,ff499c4dec1a4259,989d160f2c93] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:56:29.419 INFO 1433467 --- [alarm-detect-25,ff499c4dec1a4259,989d160f2c93] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:56:29.457 INFO 1433467 --- [alarm-detect-26,b2d5545ccc1643bd,c633c098b0e3] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316560 ~ 1710316579 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316575=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:56:29.457 INFO 1433467 --- [alarm-detect-26,b2d5545ccc1643bd,c633c098b0e3] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316560 ~ 1710316579 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316575=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:56:29.457 INFO 1433467 --- [alarm-detect-26,b2d5545ccc1643bd,c633c098b0e3] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316560 ~ 1710316579 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316575=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:56:29.910 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805743,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:30.734 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:31.0, nextRun=2024-03-13 15:56:30.0, currentDate=Wed Mar 13 15:56:30 CST 2024 2024-03-13 15:56:30.734 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999990, name=Sync tenant information, templateName=TenantSyncSchedule, scheduleRule=30 0/1 * * * ?, lastRunTime=2024-03-13 15:55:31.0, nextRunTime=2024-03-13 15:56:30.0} 2024-03-13 15:56:30.738 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Sync tenant information, nodeCount=1, taskCount=2 2024-03-13 15:56:30.738 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:30.738 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:30.738 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Sync tenant information, taskDefinitionId=999990, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:30.746 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:01.0, nextRun=2024-03-13 15:56:30.0, currentDate=Wed Mar 13 15:56:30 CST 2024 2024-03-13 15:56:30.746 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:56:01.0, nextRunTime=2024-03-13 15:56:30.0} 2024-03-13 15:56:30.747 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:56:30.747 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:30.747 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:30.869 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016756, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:30.879 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016757, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:30.881 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016756, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014405, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:30.867+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016756}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:56:30.890 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016758, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:30.898 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016757, context=Context{parallelIdx=1, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014405, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:30.877+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016757}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:56:30.911 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,b271c2444a22] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016758, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014406, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:30.888+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016758}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:30.922 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,5ac4319afbd5] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:56:30.923 INFO 1433467 --- [pool-Check obproxy13,f47a761474a84dce,ce7851ed2fff] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:56:30.923 INFO 1433467 --- [pool-Check obproxy11,f47a761474a84dce,4fbcc2fe6482] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:56:30.925 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.925 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:56:30.925 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,f57cb2525d30] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:56:30.926 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.926 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.926 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:56:30.932 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.932 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:56:30.932 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.932 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.932 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:56:30.935 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.935 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.936 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:30.937 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.937 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.937 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:30.942 INFO 1433467 --- [pool-Check obproxy13,f47a761474a84dce,ce7851ed2fff] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:56:30.943 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.943 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.944 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:30.946 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.946 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.946 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:30.947 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,5ac4319afbd5] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:56:30.952 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.952 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.952 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:30.954 INFO 1433467 --- [pool-Check obproxy11,f47a761474a84dce,4fbcc2fe6482] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:56:30.956 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,f57cb2525d30] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:56:30.959 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.959 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.960 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:30.960 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.960 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.960 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:30.965 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,b271c2444a22] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016758, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:30.969 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.969 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.969 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:30.981 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:30.981 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.982 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.982 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.985 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:56:30.991 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:30.992 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.992 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.992 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.992 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:56:30.996 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:30.997 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.997 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.997 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.997 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:56:30.998 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:30.999 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.999 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.999 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.999 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:56:31.006 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.006 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.006 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.006 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.007 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:56:31.010 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.010 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.010 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.010 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.010 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:56:31.015 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.016 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.016 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.016 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.017 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:56:31.017 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.017 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.017 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.018 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.018 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:56:31.023 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.023 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.024 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.024 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.024 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:56:31.024 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=2 2024-03-13 15:56:31.026 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:56:31.037 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.037 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.038 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.038 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.038 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:31.041 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.042 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.042 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.042 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.043 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:56:31.044 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=4, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:56:31.045 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=4, userWhitelist=% 2024-03-13 15:56:31.046 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.046 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.046 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:31.052 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=3, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:56:31.052 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.052 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=3, userWhitelist=% 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=2 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:56:31.056 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016757, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:31.069 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.069 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.069 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.069 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.070 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:56:31.072 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2024-03-13 15:56:31.075 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:56:31.093 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.093 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.094 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.094 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.095 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:31.104 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:56:31.107 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2024-03-13 15:56:31.107 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.107 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.108 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:31.113 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:56:31.115 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2024-03-13 15:56:31.115 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2024-03-13 15:56:31.115 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:56:31.120 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016756, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:31.157 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:31.158 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:31.742 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014405, status:SUCCESS 2024-03-13 15:56:31.743 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014405, status:SUCCESS 2024-03-13 15:56:31.755 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014406, status:SUCCESS 2024-03-13 15:56:31.757 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014406, status:SUCCESS 2024-03-13 15:56:31.910 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805766,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:33.197 INFO 1433467 --- [pool-ocp-schedules-15,f916a5d7c64f4d79,3c3433f80e67] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:56:33.204 INFO 1433467 --- [pool-ocp-schedules-15,f916a5d7c64f4d79,3c3433f80e67] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:56:33.204 INFO 1433467 --- [pool-ocp-schedules-15,f916a5d7c64f4d79,3c3433f80e67] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:56:33.911 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805789,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:34.158 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:34.159 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:35.911 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805812,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:36.107 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805814,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:56:36.750 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:56:36.749831+08:00, lastProcessTime=2024-03-13T15:56:26.741760+08:00 2024-03-13 15:56:36.841 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:56:36.749831+08:00 2024-03-13 15:56:36.981 INFO 1433467 --- [pool-ocp-schedules-13,9459890a6c3d4039,8131fd33b2fb] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:56:37.159 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:37.160 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:37.912 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805835,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:38.607 INFO 1433467 --- [ForkJoinPool.commonPool-worker-7,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:38.608 INFO 1433467 --- [ForkJoinPool.commonPool-worker-7,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:38.608 INFO 1433467 --- [ForkJoinPool.commonPool-worker-7,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:38.608 INFO 1433467 --- [ForkJoinPool.commonPool-worker-7,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT SVR_IP, SVR_PORT, ZONE, CPU_CAPACITY, CPU_CAPACITY_MAX, CPU_ASSIGNED, CPU_ASSIGNED_MAX, MEM_CAPACITY, MEM_CAPACITY AS MEM_CAPACITY_MAX, MEM_ASSIGNED, MEM_ASSIGNED AS MEM_ASSIGNED_MAX, LOG_DISK_CAPACITY, LOG_DISK_ASSIGNED, LOG_DISK_IN_USE, DATA_DISK_CAPACITY, DATA_DISK_IN_USE FROM oceanbase.GV$OB_SERVERS 2024-03-13 15:56:38.617 INFO 1433467 --- [ForkJoinPool.commonPool-worker-0,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:38.617 INFO 1433467 --- [ForkJoinPool.commonPool-worker-0,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:38.617 INFO 1433467 --- [ForkJoinPool.commonPool-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:38.618 INFO 1433467 --- [ForkJoinPool.commonPool-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT SVR_IP, SVR_PORT, ZONE, CPU_CAPACITY, CPU_CAPACITY_MAX, CPU_ASSIGNED, CPU_ASSIGNED_MAX, MEM_CAPACITY, MEM_CAPACITY AS MEM_CAPACITY_MAX, MEM_ASSIGNED, MEM_ASSIGNED AS MEM_ASSIGNED_MAX, LOG_DISK_CAPACITY, LOG_DISK_ASSIGNED, LOG_DISK_IN_USE, DATA_DISK_CAPACITY, DATA_DISK_IN_USE FROM oceanbase.GV$OB_SERVERS 2024-03-13 15:56:38.767 INFO 1433467 --- [pool-ocp-schedules-19,47203c1815264302,f189e4463c35] c.o.o.c.cache.helper.ObProxyQueryCache : Init obproxy cache. 2024-03-13 15:56:38.779 INFO 1433467 --- [pool-ocp-schedules-19,47203c1815264302,f189e4463c35] c.o.o.c.cache.helper.ObProxyQueryCache : Finished init obproxy cache. 2024-03-13 15:56:39.255 INFO 1433467 --- [pool-ocp-schedules-6,2ada8c597e184c1c,abb4f73366a0] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:56:39.256 INFO 1433467 --- [pool-ocp-schedules-9,3720c6b7277341c2,e62818442140] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:56:39.257 INFO 1433467 --- [pool-ocp-schedules-5,636afd317a704877,b18c0af1614c] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:56:39.257 INFO 1433467 --- [pool-ocp-schedules-6,2ada8c597e184c1c,abb4f73366a0] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:56:39.259 INFO 1433467 --- [pool-ocp-schedules-9,3720c6b7277341c2,e62818442140] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:56:39.260 INFO 1433467 --- [pool-ocp-schedules-5,636afd317a704877,b18c0af1614c] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:56:39.268 INFO 1433467 --- [pool-ocp-schedules-8,12c93531f5a54402,d70a20191936] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:56:39.268 INFO 1433467 --- [pool-Check host task for clockdiff9,12c93531f5a54402,2adeb966527d] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff20,12c93531f5a54402,64016365eb68] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff15,12c93531f5a54402,6babde3707ec] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff12,12c93531f5a54402,7d2138e338a7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff16,12c93531f5a54402,7227dbeab2dd] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff23,12c93531f5a54402,6604970ce0df] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff17,12c93531f5a54402,d254bee2f074] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:56:39.270 INFO 1433467 --- [pool-ocp-schedules-13,15c7610b60d249a8,a1c6ba6fcf4e] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:56:39.270 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:56:39.270 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:56:39.271 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:56:39.271 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:56:39.271 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:56:39.271 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:56:39.271 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:56:39.276 INFO 1433467 --- [pool-ocp-schedules-15,068f4ceb52084afb,10125273c311] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:56:39.276 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:39.277 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:39.277 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:39.278 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:39.279 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:39.279 INFO 1433467 --- [pool-ocp-schedules-4,cae93a00a1794d21,2dd0f7d06ee0] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:56:39.279 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:39.279 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:39.280 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:39.280 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:39.281 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:39.282 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:39.283 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:39.283 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:39.283 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:39.283 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805851,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:56:39.288 INFO 1433467 --- [alarm-detect-15,9c0adecadd1f4d58,f6f125a209fa] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:56:39.291 INFO 1433467 --- [alarm-detect-20,929ca6d209f14a06,f67b2a3ba7c9] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:56:39.292 INFO 1433467 --- [alarm-detect-15,9c0adecadd1f4d58,f6f125a209fa] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-ocp-schedules-18,d505c6875a514ae0,90f3581de03d] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:56:39.295 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:56:39.295 INFO 1433467 --- [alarm-detect-16,5cdda404f60f45ca,d821a25dd492] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:56:39.300 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:39.301 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:39.303 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:39.326 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:39.327 WARN 1433467 --- [pool-Check host task for clockdiff20,12c93531f5a54402,64016365eb68] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.327 ERROR 1433467 --- [pool-Check host task for clockdiff20,12c93531f5a54402,64016365eb68] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:56:39.327 WARN 1433467 --- [pool-Check host task for clockdiff20,12c93531f5a54402,64016365eb68] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.328 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:39.329 INFO 1433467 --- [alarm-detect-16,5cdda404f60f45ca,d821a25dd492] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:56:39.329 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:39.330 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:39.332 INFO 1433467 --- [ForkJoinPool-1386585-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.332 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:39.332 INFO 1433467 --- [ForkJoinPool-1386585-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.332 INFO 1433467 --- [ForkJoinPool-1386585-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.332 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:39.332 INFO 1433467 --- [ForkJoinPool-1386585-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:39.333 INFO 1433467 --- [ForkJoinPool-1386585-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:56:39.333 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:39.334 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:39.335 INFO 1433467 --- [alarm-detect-20,929ca6d209f14a06,f67b2a3ba7c9] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:56:39.335 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.336 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.336 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.336 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:39.336 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:39.339 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:56:39.341 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:39.341 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:56:39.341442683+08:00 2024-03-13 15:56:39.342 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.342 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:39.344 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.344 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:56:39.341+08:00 2024-03-13 15:56:39.350 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:56:39.351 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:56:39.352 INFO 1433467 --- [ForkJoinPool-1386585-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.344 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.352 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:56:39.352 INFO 1433467 --- [ForkJoinPool-1386585-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.352 INFO 1433467 --- [ForkJoinPool-1386585-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.352 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:56:39.352638255+08:00 2024-03-13 15:56:39.353 INFO 1433467 --- [ForkJoinPool-1386585-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:39.353 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:56:39.353405861+08:00 2024-03-13 15:56:39.354 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:39.354 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:56:39.341+08:00 and status AVAILABLE 2024-03-13 15:56:39.354 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:56:39.353+08:00 2024-03-13 15:56:39.354 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:56:39.353+08:00 2024-03-13 15:56:39.354 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:56:39.355 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:56:39.354588390+08:00 2024-03-13 15:56:39.355 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:56:39.355 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:56:39.355+08:00 2024-03-13 15:56:39.355 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.356 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:56:39.353+08:00 and status AVAILABLE 2024-03-13 15:56:39.357 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:56:39.353+08:00 and status AVAILABLE 2024-03-13 15:56:39.357 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:56:39.356802855+08:00 2024-03-13 15:56:39.357 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.357 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:56:39.357575561+08:00 2024-03-13 15:56:39.357 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:56:39.357+08:00 2024-03-13 15:56:39.358 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:56:39.355+08:00 and status AVAILABLE 2024-03-13 15:56:39.358 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:56:39.358+08:00 2024-03-13 15:56:39.358 INFO 1433467 --- [ForkJoinPool-1386585-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:56:39.359 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.359 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:56:39.357+08:00 and status AVAILABLE 2024-03-13 15:56:39.359 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:56:39.358+08:00 and status AVAILABLE 2024-03-13 15:56:39.360 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:56:39.361 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:56:39.361 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:56:39.361 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:56:39.364 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:56:39.364261836+08:00 2024-03-13 15:56:39.365 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.366 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:56:39.365+08:00 2024-03-13 15:56:39.366 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:56:39.366 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:56:39.366 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:56:39.367 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:39.367 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:56:39.365+08:00 and status AVAILABLE 2024-03-13 15:56:39.368 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:39.368 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.369 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:56:39.369 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:56:39.369 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:56:39.370 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:39.370 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:56:39.371 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:56:39.371 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.371 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:56:39.371 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:56:39.372 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:39.372 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:39.373 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:39.373 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff17,12c93531f5a54402,d254bee2f074] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 ERROR 1433467 --- [pool-Check host task for clockdiff17,12c93531f5a54402,d254bee2f074] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff16,12c93531f5a54402,7227dbeab2dd] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 ERROR 1433467 --- [pool-Check host task for clockdiff16,12c93531f5a54402,7227dbeab2dd] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff17,12c93531f5a54402,d254bee2f074] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff16,12c93531f5a54402,7227dbeab2dd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff12,12c93531f5a54402,7d2138e338a7] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 ERROR 1433467 --- [pool-Check host task for clockdiff12,12c93531f5a54402,7d2138e338a7] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff15,12c93531f5a54402,6babde3707ec] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 ERROR 1433467 --- [pool-Check host task for clockdiff15,12c93531f5a54402,6babde3707ec] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff15,12c93531f5a54402,6babde3707ec] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.418 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:56:39.418 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:56:39.418 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:56:39.418 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:39.419 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:39.419 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:39.420 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:39.422 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:39.426 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:39.431 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:39.432 WARN 1433467 --- [pool-Check host task for clockdiff12,12c93531f5a54402,7d2138e338a7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.433 WARN 1433467 --- [pool-scanner-1609800914-1,3f7a74e760e344a6,75123d1e2d4c] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:56:39.433 WARN 1433467 --- [pool-Check host task for clockdiff23,12c93531f5a54402,6604970ce0df] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.433 ERROR 1433467 --- [pool-Check host task for clockdiff23,12c93531f5a54402,6604970ce0df] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:56:39.434 WARN 1433467 --- [pool-Check host task for clockdiff23,12c93531f5a54402,6604970ce0df] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.434 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.434 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.438 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.440 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.440 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.440 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.441 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:56:39.442 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.456 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.457 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.459 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:56:39.459 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.462 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.462 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.462 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.463 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.463 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.463 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:39.463 WARN 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:56:39.463 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.463 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:56:39.463 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.464 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:56:39.464 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:56:39.464 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:39.469 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:56:39.471 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:39.472 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:39.475 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.477 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:56:39.490 WARN 1433467 --- [pool-Check host task for clockdiff9,12c93531f5a54402,2adeb966527d] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.490 ERROR 1433467 --- [pool-Check host task for clockdiff9,12c93531f5a54402,2adeb966527d] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:56:39.490 WARN 1433467 --- [pool-Check host task for clockdiff9,12c93531f5a54402,2adeb966527d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.490 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:56:39.491 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:56:39.491 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:56:39.492 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:39.492 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:56:39.497 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:56:39.501 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:56:39.502 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.503 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:56:39.503 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:56:39.504 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:56:39.506 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.507 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.507 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.507 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.508 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:39.509 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:56:39.511 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:56:39.514 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:56:39.514 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.515 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:39.516 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:39.518 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.519 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.520 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:56:39.520 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.521 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.530 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:56:39.533 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.534 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:56:39.536 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:56:39.541 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:56:39.542 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:56:39.542 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:56:39.547 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:56:39.548 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:56:39.553 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:56:39.553 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:56:39.558 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:56:39.561 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:56:39.561 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:56:39.564 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.564 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.570 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.575 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.578 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.578 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:56:39.584 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:56:39.595 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:56:39.598 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:56:39.600 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:56:39.604 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:56:39.606 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:56:39.613 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:56:39.616 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:56:39.617 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.618 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:56:39.620 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:56:39.624 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:56:39.642 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:56:39.649 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.s.o.o.f