2024-03-13 15:54:41.887 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804482,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:43.108 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:43.113 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:43.655 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,6d436924921a4747,6c9b1bd2cacf] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.142, traceId=6d436924921a4747, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:54:43.660 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,6d436924921a4747,6c9b1bd2cacf] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.142, traceId=6d436924921a4747, duration=6 ms] 2024-03-13 15:54:43.887 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804505,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:45.888 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804528,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:46.102 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804532,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:54:46.113 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:46.114 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:46.748 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:54:46.747757+08:00, lastProcessTime=2024-03-13T15:54:36.746676+08:00 2024-03-13 15:54:46.864 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:54:46.747757+08:00 2024-03-13 15:54:46.910 INFO 1433467 --- [pool-ocp-schedules-13,9f1fa9f82daf4bae,cdaa10dd1e44] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:54:47.888 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804552,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:47.908 INFO 1433467 --- [pool-clean-time-series-data15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:54:47.924 INFO 1433467 --- [pool-print-time-series-interval15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316487: {1710316380=1394, 1710316395=1394, 1710316410=1394, 1710316425=1394, 1710316440=1394, 1710316455=1394, 1710316470=1394, 1710316485=60}, tsDataMapSize: 1122696, readyDataCacheSize: 8288 2024-03-13 15:54:47.948 INFO 1433467 --- [pool-clean-time-series-data60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:54:47.949 INFO 1433467 --- [pool-print-time-series-interval60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316487: {1710316320=32, 1710316380=34, 1710316440=6}, tsDataMapSize: 17768, readyDataCacheSize: 18488 2024-03-13 15:54:49.114 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:49.115 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:49.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804569,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:54:49.321 WARN 1433467 --- [pool-scanner-1609791898-1,e615076dce214056,570005faba11] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:54:49.412 INFO 1433467 --- [alarm-detect-17,f1a954638fdb409a,0eb0dc68bffc] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000911, data is empty. 2024-03-13 15:54:49.415 INFO 1433467 --- [alarm-detect-17,f1a954638fdb409a,0eb0dc68bffc] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000911, data is empty. 2024-03-13 15:54:49.456 INFO 1433467 --- [alarm-detect-11,aaaa33118a1e4658,7de15b276b07] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316455 ~ 1710316479 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316470=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:54:49.456 INFO 1433467 --- [alarm-detect-11,aaaa33118a1e4658,7de15b276b07] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316455 ~ 1710316479 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316470=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:54:49.888 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804576,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:51.889 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804599,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:52.116 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:52.117 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:52.719 INFO 1433467 --- [pool-ocp-schedules-3,d61bd653daa749fc,bdc27671a07c] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:54:52.726 INFO 1433467 --- [pool-ocp-schedules-3,d61bd653daa749fc,bdc27671a07c] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:54:52.726 INFO 1433467 --- [pool-ocp-schedules-3,d61bd653daa749fc,bdc27671a07c] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:54:53.889 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804622,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:55.117 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:55.118 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:55.688 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2024-03-13 15:54:55.688 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=609459010050985984 2024-03-13 15:54:55.699 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2024-03-13 15:54:55.699 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624101396131614721 2024-03-13 15:54:55.707 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2024-03-13 15:54:55.707 INFO 1433467 --- [pool-ocp-schedules-16,31e4829cb47f4a56,431b8aa1a26c] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624425259050733570 2024-03-13 15:54:55.889 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804645,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:56.103 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804649,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:54:56.125 INFO 1433467 --- [pool-Check obproxy13,18b3329f433b446c,9509184b4a96] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:54:56.127 INFO 1433467 --- [pool-Check obproxy14,18b3329f433b446c,0aa0a18a242f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:54:56.127 INFO 1433467 --- [pool-Check obproxy8,18b3329f433b446c,6a847e76f048] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:54:56.128 INFO 1433467 --- [pool-Check obproxy12,18b3329f433b446c,3d7db83b0c7f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:54:56.746 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:54:56.745596+08:00, lastProcessTime=2024-03-13T15:54:46.747757+08:00 2024-03-13 15:54:56.828 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:54:56.745596+08:00 2024-03-13 15:54:56.916 INFO 1433467 --- [pool-ocp-schedules-13,9a736d98159a4fc2,172b642b20e7] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:54:57.890 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804668,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:54:58.118 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:54:58.119 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:54:59.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804685,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:54:59.281 INFO 1433467 --- [alarm-detect-11,cdc71058cc8e4188,6e4abc818365] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000911, data is empty. 2024-03-13 15:54:59.282 INFO 1433467 --- [alarm-detect-17,9afb90b55ab14982,14a686aeecf6] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000911, data is empty. 2024-03-13 15:54:59.283 INFO 1433467 --- [alarm-detect-25,39bab5caafb84041,f17cb15cd0cb] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000911, data is empty. 2024-03-13 15:54:59.286 INFO 1433467 --- [alarm-detect-11,cdc71058cc8e4188,6e4abc818365] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000911, data is empty. 2024-03-13 15:54:59.287 INFO 1433467 --- [alarm-detect-17,9afb90b55ab14982,14a686aeecf6] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000911, data is empty. 2024-03-13 15:54:59.287 INFO 1433467 --- [alarm-detect-25,39bab5caafb84041,f17cb15cd0cb] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000911, data is empty. 2024-03-13 15:54:59.329 WARN 1433467 --- [pool-scanner-1609792550-1,1c87fe138a164bf1,5eabab7754e7] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:54:59.442 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316470 ~ 1710316489 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316485=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:54:59.442 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316470 ~ 1710316489 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316485=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:54:59.442 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316470 ~ 1710316489 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316485=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:54:59.442 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316470 ~ 1710316489 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316485=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:54:59.442 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316470 ~ 1710316489 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316485=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:54:59.443 INFO 1433467 --- [alarm-detect-20,5b57b564164d46c7,d501c21e8986] c.o.o.a.core.detect.AlarmRuleDetector : detectByMetric done, alarmType=host_ntp_service_not_exist, needSendingCount=1, detectAlarmsCount=1, matchedTargets=[alarm_template_id=3:host=10.10.10.148-2, alarm_template_id=3:host=10.10.10.147-2, alarm_template_id=3:host=10.10.10.146-2, alarm_template_id=3:host=10.10.10.145-2, alarm_template_id=3:host=10.10.10.144-2] 2024-03-13 15:54:59.891 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804692,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:00.497 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:00.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.497 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=38, name=Detect alarms on schedule, templateName=BackupScheduleAlarmTemplateFactory, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:54:00.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.500 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Detect alarms, nodeCount=1, taskCount=1 2024-03-13 15:55:00.501 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.501 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Detect alarms, taskDefinitionId=38, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.508 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:50:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.509 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999977, name=DiagFuncUpdateSchedule, templateName=DiagFuncUpdateSchedule, scheduleRule=0 */5 * * * *, lastRunTime=2024-03-13 15:50:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.510 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=DiagFuncUpdateSchedule, nodeCount=1, taskCount=1 2024-03-13 15:55:00.510 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.510 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=DiagFuncUpdateSchedule, taskDefinitionId=999977, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.514 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:50:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.514 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999978, name=HighRiskSqlSchedule, templateName=HighRiskSqlSchedule, scheduleRule=0 */5 * * * *, lastRunTime=2024-03-13 15:50:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.539 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=HighRiskSqlAuditStatTaskSchedule, nodeCount=1, taskCount=1 2024-03-13 15:55:00.539 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.539 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=HighRiskSqlAuditStatTaskSchedule, taskDefinitionId=999978, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.543 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:50:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.543 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999980, name=Collect all tenants mini compactions, templateName=CollectTenantMiniCompactionSchedule, scheduleRule=0 */5 * * * *, lastRunTime=2024-03-13 15:50:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.545 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all tenants mini compactions, nodeCount=1, taskCount=1 2024-03-13 15:55:00.545 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.545 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all tenants mini compactions, taskDefinitionId=999980, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.549 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:00.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.549 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999981, name=Collect all tenants major compactions, templateName=CollectTenantCompactionSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:54:00.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.550 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all tenants major compactions, nodeCount=1, taskCount=1 2024-03-13 15:55:00.550 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.550 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all tenants major compactions, taskDefinitionId=999981, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.554 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:00.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.554 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999982, name=Collect SlowSql to SqlAggregateData for alarm, templateName=CollectSlowSqlSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:54:00.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.559 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect SlowSql to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:55:00.559 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.559 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect SlowSql to SqlAggregateData for alarm, taskDefinitionId=999982, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.564 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.564 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999983, name=Collect trans stat to SqlAggregateData for alarm, templateName=CollectTransSatatSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:54:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.566 INFO 1433467 --- [pool-prometheus-minute-schedule-1,fbce573c7eb84c5f,f6cfe8b80370] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1710316560, interval=60 2024-03-13 15:55:00.567 INFO 1433467 --- [pool-prometheus-cache-monitor1,04c2f32a4ca04484,262ae173d066] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":12072,"minuteWriteQueue":0,"minuteReadCacheSize":12328,"secondWriteCacheSize":3332,"seriesIdKeySize":67930,"secondWriteQueue":0,"minuteWriteCacheSize":12393} 2024-03-13 15:55:00.568 INFO 1433467 --- [pool-prometheus-cache-monitor1,04c2f32a4ca04484,262ae173d066] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{go_memory_classes_heap_stacks_bytes=15, ob_sysstat_40012=19, ob_server_resource_memory_assigned_bytes=13, ob_sysstat_40011=19, ob_sysstat_40010=19, ob_unit_num=13, ob_database_disk_required_size=35, go_memstats_mspan_sys_bytes=15, go_memstats_stack_sys_bytes=15, ob_tenant_request_queue_total_size=18, jvm_gc_memory_allocated_bytes_total=3, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=18, ob_sysstat_40019=19, observer_fd_count=13, ob_sysstat_40018=19, processor_process_metrics_total=21, jvm_gc_max_data_size_bytes=3, go_memstats_frees_total=15, sql_audit_input_collect_data_jumped_total=8, host_available=64, mysql_output_write_sql_failed_count=17, ob_database_disk_data_size=34, ob_system_event_detail_total_timeouts=763, ob_tenant_memstore_freeze_cnt=14, ob_latch_sleep_total=312, ob_tenant_unit_min_memory=25, ob_sysstat_60087=19, ob_memstore_total_bytes=20, metric_cardinality_count=12, process_resident_memory_bytes=15, jvm_threads_states_threads=8, ob_cache_size_bytes=34, ob_tenant_unit_max_memory=25, input_collect_metrics_total=26, ob_sysstat_60083=19, ob_sysstat_50039=19, ob_sysstat_50038=19, node_disk_io_time_weighted_seconds_total=33, process_open_fds=15, ob_sysstat_50037=19, process_memory_rss_bytes=33, ob_server_resource_cpu_assigned_percent=13, ob_sysstat_50030=19, go_memory_classes_profiling_buckets_bytes=15, go_memstats_heap_inuse_bytes=15, ob_sysstat_40118=19, ob_sysstat_40117=19, ob_sysstat_40116=19, node_memory_memfree_bytes=26, ob_sysstat_40005=19, process_exists=43, ob_sysstat_40004=19, ob_sysstat_40003=19, ob_sysstat_40002=19, ob_sysstat_40001=19, go_gc_duration_seconds=20, go_memory_classes_metadata_mspan_inuse_bytes=15, ob_server_resource_disk_bytes=13, ob_sysstat_40000=19, odp_current_session=26, process_thread_count=33, ob_sysstat_50040=19, ob_sysstat_40009=19, ob_sysstat_40008=19, ob_disk_free_bytes=13, node_filesystem_avail_bytes=72, ob_sysstat_40007=19, ob_sysstat_40006=19, ob_backup_storage_capacity_bytes=650, ob_unit_config_max_iops=21, ob_plan_cache_memory_bytes=18, node_network_transmit_errs_total=28, go_gc_duration_seconds_count=15, monagent_oceanbase_plan_explain_size=4, ob_sysstat_50011=19, ob_sysstat_50010=19, node_disk_read_bytes_total=33, ob_compaction_is_suspended=19, mysql_output_write_sql_milliseconds_summary_count=17, go_memstats_lookups_total=15, mysql_output_write_sql_count=17, process_cpu_percent=33, ob_sysstat_60057=19, host_agent_process_status=18, ob_server_resource_cpu=13, ob_tenant_server_data_index_row_count=20, go_memstats_alloc_bytes=15, go_sched_gomaxprocs_threads=15, node_cpu_seconds_total_native=38, ob_sysstat_140002=19, ob_sysstat_140005=19, ob_sysstat_140006=19, ob_sysstat_140003=19, ob_tenant_resource_min_memory=18, ob_memstore_freeze_times=20, ob_sysstat_50022=19, jvm_gc_pause_seconds_count=8, ob_sysstat_50021=19, jvm_gc_overhead_percent=3, go_memory_classes_heap_unused_bytes=15, ob_cluster_status_check=13, jvm_memory_committed_bytes=10, node_net_bandwidth_bps=29, monagent_oceanbase_db_pool_idle=4, ob_tenant_server_data_index_data_size=20, go_gc_heap_allocs_by_size_bytes_total_count=15, go_memstats_heap_idle_bytes=15, go_gc_cycles_forced_gc_cycles_total=15, go_gc_pauses_seconds_total_bucket=26, ob_server_resource_memory_bytes=13, ob_tenant_assigned_mem_assigned=18, ob_tenant_memstore_minor_freeze_times=14, ob_tenant_server_required_size=22, ob_sysstat_60031=19, process_fd_count=33, ob_tenant_unit_max_cpu=25, ob_latch_wait_seconds_total=312, go_memory_classes_metadata_mcache_free_bytes=15, node_memory_buffers_bytes=26, ob_sysstat_50009=19, go_memstats_heap_released_bytes=15, node_network_receive_bytes_total=28, ob_sysstat_50005=19, ob_sysstat_50008=19, ob_sysstat_50001=19, ob_sysstat_50004=19, exposer_request_latencies_count=6, ob_sysstat_50000=19, ob_session_active_num=18, mysql_output_context_timeout_discard_metrics_count=9, go_memory_classes_metadata_mcache_inuse_bytes=15, node_disk_read_time_seconds_total=33, ob_sysstat_60021=19, ob_sysstat_60020=19, ob_sysstat_60024=19, ob_compaction_frozen_version=19, ob_sysstat_60023=19, metric_count=15, ob_sysstat_60022=19, obproxysys_connectable=1556, ob_tenant_assigned_cpu_assigned=18, mysql_output_write_sql_retry_count=17, ob_tenant_unit_min_cpu=25, ob_unit_config_max_cpu=21, http_request_milliseconds_summary_count=75, monagent_oceanbase_audit_timeinfo_cache_size=4, http_request_milliseconds_summary=78, ob_latch_immediate_miss_total=312, ob_latch_spin_get_total=312, ob_log_level_count=17, monagent_pipeline_report_metrics_total=16, exposer_transferred_bytes_total=6, ob_tenant_context_memory_count=841, node_disk_reads_completed_total=33, ob_sysstat_60019=19, ob_sysstat_60003=19, ob_sysstat_60002=19, ob_sysstat_60001=19, ob_sysstat_60000=19, ob_latch_miss_total=312, go_sched_latencies_seconds_count=15, ocp_meta_db_health=5, ob_disk_total_bytes=13, go_memstats_heap_objects=15, go_memstats_mcache_inuse_bytes=15, ob_sysstat_60005=19, ob_sysstat_60004=19, go_memory_classes_other_bytes=15, ob_memstore_active_bytes=20, go_memstats_alloc_bytes_total=15, ob_tenant_server_partition_replica_count=21, process_count=34, ob_system_event_detail_total_waits=763, odp_used_connections=25, backup_storage_capacity_used_percent=8, go_gc_heap_frees_objects_total=15, go_memory_classes_metadata_mspan_free_bytes=15, ob_disk_log_used_bytes=13, ob_compaction_is_error=19, go_gc_limiter_last_enabled_gc_cycle=15, monagent_pipeline_execute_total=24, node_filesystem_files=86, process_start_time_seconds=15, go_memstats_heap_alloc_bytes=15, ob_tenant_resource_max_cpu=18, ntp_service_available=26, go_memstats_mspan_inuse_bytes=15, mysql_output_write_sql_milliseconds_summary=20, go_gc_heap_frees_by_size_bytes_total_count=15, monagent_pipeline_execute_seconds_total=24, ob_sysstat_80001=19, go_memstats_sys_bytes=15, go_memory_classes_heap_free_bytes=15, system_load_average_1m=3, monagent_oceanbase_audit_stat_cache_size=4, ob_tenant_memory_used_bytes=22, obproxy_server_max_connections=7, ob_sysstat_80002=19, go_memstats_gc_cpu_fraction=15, ob_sysstat_80003=19, monagent_oceanbase_audit_cache_size=4, ob_tenant_context_memory_used_bytes=841, ob_sysstat_80007=19, node_disk_writes_completed_total=33, ob_sysstat_80008=19, ob_sysstat_80009=19, go_sched_goroutines_goroutines=15, go_memory_classes_metadata_other_bytes=15, go_gc_cycles_total_gc_cycles_total=15, system_cpu_usage=3, process_cpu_usage=3, node_disk_written_bytes_total=33, monagent_oceanbase_ash_cache_size=4, go_goroutines=15, ob_sysstat_80023=19, exposer_request_latencies_sum=6, ob_unit_config_min_cpu=21, process_virtual_memory_max_bytes=15, go_gc_heap_frees_bytes_total=15, ob_tenant500_memory_hold_bytes=13, exposer_scrapes_total=6, go_gc_heap_allocs_by_size_bytes_total_bucket=27, ob_sysstat_80025=19, ob_sysstat_80026=19, oceanbase_connectivity=15, node_memory_sreclaimable_bytes=26, go_sched_latencies_seconds_sum=15, tcp_retrans=25, exporter_export_metrics_bytes_total=26, node_filesystem_size_bytes=72, http_server_requests_seconds_count=203, ob_compaction_frozen_time=19, monagent_input_sql_duration_seconds_sum=55, log_stream_degraded_count=18, ob_plan_cache_hit_total=18, node_filesystem_readonly=72, ob_sysstat_80040=19, ob_sysstat_80041=19, io_await=52, go_gc_heap_allocs_objects_total=15, go_memstats_mallocs_total=15, ob_unit_config_max_memory_bytes=21, go_gc_stack_starting_size_bytes=15, io_util=52, ob_sysstat_200002=19, ob_sysstat_200001=19, ob_tenant_disk_v4_data_size=18, go_sched_latencies_seconds_bucket=26, ob_tenant_assigned_cpu_total=18, ob_tenant_server_available_disk_free=19, node_disk_write_time_seconds_total=33, cpu_count=25, output_write_metrics_total=9, ob_cluster_sync=15, go_gc_heap_objects_objects=15, odp_transaction_total=16, ob_system_event_detail_time_waited_us=763, process_max_fds=15, http_status_count=20, ob_sysstat_80057=19, ocp_alarm_detect_duration_ms_count=115, go_gc_heap_allocs_by_size_bytes_total_sum=15, go_memstats_mcache_sys_bytes=15, ob_tenant_disk_v4_log_size=18, go_cgo_go_to_c_calls_calls_total=15, ocp_alarm_detect_duration_ms_sum=115, jvm_threads_live_threads=3, node_memory_cached_bytes=26, ob_table_num=16, ob_sysstat_10000=19, ob_sysstat_10006=19, ob_sysstat_10005=19, ob_sysstat_10002=19, ob_sysstat_10001=19, go_memstats_last_gc_time_seconds=15, ob_sysstat_10004=19, ob_sysstat_10003=19, monitor_exporter_avaliable=62, log_tailer_count=15, ob_server_num=25, ob_session_all_num=18, ob_system_event_time_waited=24, ob_tenant_resource_min_cpu=18, monagent_plugin_execute_seconds_total=39, go_gc_cycles_automatic_gc_cycles_total=15, go_gc_heap_allocs_bytes_total=15, go_memstats_gc_sys_bytes=15, ob_latch_get_total=312, go_info=16, log_tailer_processed_file_count=21, node_load15=24, ob_disk_log_assigned_bytes=13, go_memory_classes_heap_released_bytes=15, jvm_memory_max_bytes=10, ob_unit_config_min_iops=21, ob_sysstat_30080=19, ob_sysstat_30082=19, ob_sysstat_30081=19, ob_sysstat_30084=19, ob_sysstat_30083=19, ob_sysstat_30086=19, ob_sysstat_30085=19, ob_tenant_log_disk_total_bytes=18, ob_memstore_freeze_trigger_bytes=20, ob_waitevent_wait_seconds_total=18, ob_server_stopped_duration_seconds=15, go_memstats_next_gc_bytes=15, node_filesystem_files_free=86, ob_memtable_max_snapshot_duration_seconds=18, monagent_oceanbase_audit_unique_key_cache_size=4, process_virtual_memory_bytes=15, ob_sysstat_20002=19, net_connectivity=15, ob_sysstat_20001=19, monagent_input_sql_duration_seconds_bucket=65, ob_latch_immediate_get_total=312, node_cpu_seconds_total=32, ob_sysstat_20000=19, ob_index_error_num=13, ob_tenant_server_available_disk_total=19, ob_tenant_memory_hold_bytes=22, ob_sysstat_30066=19, go_memstats_buck_hash_sys_bytes=15, mysql_output_write_sql_milliseconds_summary_sum=17, ob_tenant_disk_total_size=18, odp_sql_request_total=53, mysql_output_write_task_size=17, exporter_export_metrics_total=26, ob_disk_log_total_bytes=13, system_cpu_count=3, ob_sysstat_130004=19, process_uptime_seconds=33, node_ntp_offset_seconds=25, go_memstats_heap_sys_bytes=15, ob_sysstat_140012=19, ob_sysstat_140013=19, jvm_gc_pause_seconds_max=8, ob_tenant_request_queue_large_size=18, ob_sysstat_130000=19, go_memstats_other_sys_bytes=15, ob_sysstat_130001=19, jvm_memory_used_bytes=10, ob_sysstat_130002=19, go_threads=15, http_request_milliseconds_summary_sum=75, http_server_requests_seconds_sum=203, ob_tenant_assigned_mem_total=18, node_memory_memtotal_bytes=26, node_network_transmit_drop_total=28, go_gc_pauses_seconds_total_count=15, exposer_request_latencies=9, ob_waitevent_wait_total=18, go_gc_heap_goal_bytes=15, ob_tenant_log_disk_used_bytes=18, jvm_gc_memory_promoted_bytes_total=3, odp_sql_cost_total=51, ob_sysstat_30011=19, ob_sysstat_30010=19, ob_sysstat_30013=19, ob_sysstat_30012=19, ob_tenant500_memory_used_bytes=13, node_network_transmit_bytes_total=28, node_network_receive_drop_total=28, go_gc_heap_tiny_allocs_objects_total=15, go_gc_duration_seconds_sum=15, ob_server_resource_memory_assigned_percent=13, jvm_gc_pause_seconds_sum=8, go_memory_classes_heap_objects_bytes=15, node_network_receive_errs_total=28, observer_ulimit_max_fd_count=13, ob_plan_cache_access_total=18, ob_tenant_server_row_count=22, monagent_oceanbase_plan_cache_size=4, ob_tenant_server_data_size=22, monagent_oceanbase_audit_sample_cache_size=4, go_gc_heap_frees_by_size_bytes_total_sum=15, ob_sysstat_40030=19, node_load1=24, process_cpu_seconds_total=15, ob_sysstat_120001=19, go_memory_classes_total_bytes=15, ob_sysstat_120002=19, ob_sysstat_120000=19, ob_tenant_resource_max_memory=18, ob_sysstat_120009=19, go_memstats_stack_inuse_bytes=15, ob_sysstat_120008=19, node_load5=24, ob_sysstat_120005=19, go_memory_classes_os_stacks_bytes=15, ob_sysstat_120006=19, jvm_threads_peak_threads=3, monagent_input_sql_duration_seconds_count=55, ob_tenant_context_memory_hold_bytes=841, obproxy_memory_limit_bytes=7, ob_compaction_last_version=19, ob_sysstat_30000=19, ob_server_resource_cpu_assigned=13, ob_sysstat_30002=19, ob_sysstat_30001=19, monagent_oceanbase_plan_timeinfo_cache_size=4, monagent_plugin_execute_total=39, ob_sysstat_30006=19, ob_sysstat_30005=19, ob_sysstat_30008=19, ob_sysstat_30007=19, ob_sysstat_30009=19, go_gc_heap_frees_by_size_bytes_total_bucket=27, obproxy_client_max_connections=7, ob_compaction_current_timestamp=19, ob_clog_max_sync_delay_seconds=20, go_gc_pauses_seconds_total_sum=15, ob_compaction_start_time=19} 2024-03-13 15:55:00.569 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect trans stat to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:55:00.569 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.569 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect trans stat to SqlAggregateData for alarm, taskDefinitionId=999983, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.572 INFO 1433467 --- [pool-prometheus-minute-schedule-1,fbce573c7eb84c5f,f6cfe8b80370] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=18, collectTaskCount=18 2024-03-13 15:55:00.572 INFO 1433467 --- [pool-prometheus-minute-schedule-1,fbce573c7eb84c5f,f6cfe8b80370] c.o.ocp.monitor.OcpMonitorManager : Schedule interval metric too slow, from=1710316500, count=18, elapsed=1708606184072 2024-03-13 15:55:00.580 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.581 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999984, name=Get agent alarms, templateName=GetAgentAlarmsSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:54:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.587 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Get agent alarms, nodeCount=1, taskCount=1 2024-03-13 15:55:00.587 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.587 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Get agent alarms, taskDefinitionId=999984, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.591 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:50:01.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.591 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999985, name=SqlParseSchedule, templateName=SqlParseSchedule, scheduleRule=0 */5 * * * *, lastRunTime=2024-03-13 15:50:01.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.601 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.p.s.internal.SqlParseTaskSchedule : get Clusters for parse:[1, 2] 2024-03-13 15:55:00.611 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.p.s.internal.SqlParseTaskSchedule : HighRiskSql Parse is ready 2024-03-13 15:55:00.611 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=SqlParseTaskSchedule, nodeCount=1, taskCount=1 2024-03-13 15:55:00.611 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.611 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=SqlParseTaskSchedule, taskDefinitionId=999985, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.616 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:50:02.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.616 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999986, name=Check all cluster configUrl, templateName=CheckClusterConfigUrlSchedule, scheduleRule=0 */5 * * * ?, lastRunTime=2024-03-13 15:50:02.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.618 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=CheckClusterConfigUrlSchedule, nodeCount=1, taskCount=1 2024-03-13 15:55:00.618 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.618 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=CheckClusterConfigUrlSchedule, taskDefinitionId=999986, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.625 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:31.0, nextRun=2024-03-13 15:55:00.0, currentDate=Wed Mar 13 15:55:00 CST 2024 2024-03-13 15:55:00.625 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:54:31.0, nextRunTime=2024-03-13 15:55:00.0} 2024-03-13 15:55:00.626 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:55:00.626 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:00.626 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:00.738 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016732, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.753 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016733, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.756 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016732, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014383, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.734+08:00, sub_task_instance_name=Detect alarms of backup, sub_task_instance_id=10016732}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.764 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016734, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.767 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016733, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014384, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.751+08:00, sub_task_instance_name=Update diag function, sub_task_instance_id=10016733}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.777 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016735, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.777 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.778 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016734, context=Context{parallelIdx=0, stringMap={subtask_splitter=tenant_ids, task_instance_id=10014385, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.762+08:00, sub_task_instance_name=High Risk Sql Audit Stat Task, sub_task_instance_id=10016734}, listMap={cluster_ids=[[1,2]], tenant_ids=[[2,4]]}}, executor=10.10.10.148 2024-03-13 15:55:00.779 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.t.HighRiskSqlAuditStatTask : start to diagnose[2, 4] 2024-03-13 15:55:00.778 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:00.780 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.786 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.788 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:55:00.793 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_1_2currentTimeUs:1710316500779000 2024-03-13 15:55:00.794 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_1_2diagnosisStartTimeUs:1710316201199000 2024-03-13 15:55:00.805 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,41d527ac6c6c] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016735, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014386, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.775+08:00, sub_task_instance_name=Collect all tenants mini compactions, sub_task_instance_id=10016735}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.805 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,41d527ac6c6c] t.b.t.CollectAllTenantMiniCompactionTask : Begin to collect all tenants mini compaction info 2024-03-13 15:55:00.808 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016736, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.818 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016737, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.818 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect mini compaction info 2024-03-13 15:55:00.818 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect mini compaction info 2024-03-13 15:55:00.822 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,6fbd46b119bd] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016736, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014387, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.806+08:00, sub_task_instance_name=Collect all tenants major compactions, sub_task_instance_id=10016736}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.822 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,6fbd46b119bd] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2024-03-13 15:55:00.825 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016738, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.825 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_1_2totalLength:0 2024-03-13 15:55:00.828 INFO 1433467 --- [pool-subtask-executor14,f47a761474a84dce,5182fb0b3024] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016737, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014388, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.816+08:00, sub_task_instance_name=Collect SlowSql to SqlAggregateData for alarm, sub_task_instance_id=10016737}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.829 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_2_4currentTimeUs:1710316500827000 2024-03-13 15:55:00.829 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_2_4diagnosisStartTimeUs:1710316201173000 2024-03-13 15:55:00.833 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:55:00.833 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:55:00.833 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:55:00.833 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:55:00.834 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.p.s.i.HighRiskSqlInspectService : taskName:high_risk_sql_audit_diag_2_4totalLength:0 2024-03-13 15:55:00.836 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016739, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.839 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:00.839 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.839 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.840 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,a289468e6e06] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016738, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014389, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.823+08:00, sub_task_instance_name=Collect trans stat to SqlAggregateData for alarm, sub_task_instance_id=10016738}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.840 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.840 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.841 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:55:00.841 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.841 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.842 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.842 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.842 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:00.842 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:55:00.842 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.843 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.843 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.843 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,1e1152eaf638] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016734, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.844 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:55:00.845 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402011840869, broadcast scn:1710266402011840869, saved frozen scn:1710266402011840869 2024-03-13 15:55:00.845 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,5bb80af34ebf] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:55:00.846 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266403004770377, broadcast scn:1710266403004770377, saved frozen scn:1710266403004770377 2024-03-13 15:55:00.847 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,7053a20fb748] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:55:00.848 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266400687471944, broadcast scn:1710266400687471944, saved frozen scn:1710266400687471944 2024-03-13 15:55:00.848 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,51a2f2945c84] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:55:00.849 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.850 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.850 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.850 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402269282986, broadcast scn:1710266402269282986, saved frozen scn:1710266402269282986 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,b263ae60a916] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016740, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.852 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, MIN(unix_timestamp(`start_time`)*1000000) AS min_start_time, MAX(unix_timestamp(`finish_time`)*1000000) AS max_finish_time, SUM(occupy_size) AS occupy_size, SUM(total_row_count) AS total_row_count, COUNT(1) AS tablet_count FROM GV$OB_TABLET_COMPACTION_HISTORY WHERE type = 'MINI_MERGE' AND unix_timestamp(`finish_time`)*1000000 > ? AND unix_timestamp(`finish_time`)*1000000 <= ? GROUP BY tenant_id;, args: [1710315300000000, 1710316500000000] 2024-03-13 15:55:00.853 INFO 1433467 --- [pool-subtask-executor29,f47a761474a84dce,48006746c3a2] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016739, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014390, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.831+08:00, sub_task_instance_name=Get all agent alarms, sub_task_instance_id=10016739}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.855 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,6fbd46b119bd] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2024-03-13 15:55:00.858 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,a289468e6e06] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster1, trans stat result is empty. 2024-03-13 15:55:00.858 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,a289468e6e06] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster, trans stat result is empty. 2024-03-13 15:55:00.858 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,6fbd46b119bd] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016736, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.858 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.858 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.859 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, MIN(unix_timestamp(`start_time`)*1000000) AS min_start_time, MAX(unix_timestamp(`finish_time`)*1000000) AS max_finish_time, SUM(occupy_size) AS occupy_size, SUM(total_row_count) AS total_row_count, COUNT(1) AS tablet_count FROM GV$OB_TABLET_COMPACTION_HISTORY WHERE type = 'MINI_MERGE' AND unix_timestamp(`finish_time`)*1000000 > ? AND unix_timestamp(`finish_time`)*1000000 <= ? GROUP BY tenant_id;, args: [1710315300000000, 1710316500000000] 2024-03-13 15:55:00.860 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016741, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.870 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,a289468e6e06] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016738, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.871 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016742, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:00.876 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016740, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014391, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.850+08:00, sub_task_instance_name=Sql Parse Task, sub_task_instance_id=10016740}, listMap={cluster_ids=[[1,2]]}}, executor=10.10.10.148 2024-03-13 15:55:00.876 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.p.sql.internal.task.SqlParseTask : parse sql for cluster:1 2024-03-13 15:55:00.879 INFO 1433467 --- [pool-subtask-executor14,f47a761474a84dce,5182fb0b3024] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for obcluster1, save 1 to SqlAggregateData 2024-03-13 15:55:00.881 INFO 1433467 --- [pool-subtask-executor14,f47a761474a84dce,5182fb0b3024] c.o.o.p.s.i.alarm.SlowSqlDataSource : query sqlCount of obCluster obcluster, sql count is 0. 2024-03-13 15:55:00.881 INFO 1433467 --- [pool-subtask-executor14,f47a761474a84dce,5182fb0b3024] c.o.o.p.s.i.alarm.SlowSqlDataSource : query SlowSqlData of obCluster obcluster, result is empty. 2024-03-13 15:55:00.882 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,894e0c37e2cb] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016741, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014392, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.858+08:00, sub_task_instance_name=Check root service , sub_task_instance_id=10016741}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.882 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:00.883 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,894e0c37e2cb] c.o.o.s.t.b.c.AbstractAllClusterTask : Check root service executed completed, tryCount=2, successCount=0 2024-03-13 15:55:00.883 INFO 1433467 --- [pool-35608-thread-1,f47a761474a84dce,78ece07259c5] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect mini compaction info 2024-03-13 15:55:00.920 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.884 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,02cc2db91f20] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016742, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014393, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:00.870+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016742}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:00.923 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.926 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.928 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:55:00.930 INFO 1433467 --- [pool-Check obproxy10,f47a761474a84dce,fefdadc2e595] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:55:00.930 INFO 1433467 --- [pool-Check obproxy5,f47a761474a84dce,5ce32a29443d] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:55:00.930 INFO 1433467 --- [pool-Check obproxy15,f47a761474a84dce,d2d852e369d7] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:55:00.932 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,93b21b0dfde5] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:55:00.933 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2024-03-13 15:55:00.933 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.934 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.934 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.934 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [obconfig_url] 2024-03-13 15:55:00.934 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1,obcluster1,1002,1710288000000000,2000,30000000 2024-03-13 15:55:00.885 INFO 1433467 --- [pool-subtask-executor14,f47a761474a84dce,5182fb0b3024] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016737, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.920 INFO 1433467 --- [pool-35608-thread-2,f47a761474a84dce,a6803250222f] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect mini compaction info 2024-03-13 15:55:00.935 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2024-03-13 15:55:00.935 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2024-03-13 15:55:00.923 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,894e0c37e2cb] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016741, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.924 INFO 1433467 --- [pool-subtask-executor29,f47a761474a84dce,48006746c3a2] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016739, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.936 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:00.938 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:00.939 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:00.939 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:00.941 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.perf.sql.internal.SqlParseService : parse 0 sqls for tenant:ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL} 2024-03-13 15:55:00.941 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.p.sql.internal.task.SqlParseTask : parse sql for cluster:2 2024-03-13 15:55:00.941 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:00.941 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:00.942 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.943 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.943 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.943 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [obconfig_url] 2024-03-13 15:55:00.944 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=6 2024-03-13 15:55:00.944 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:55:00.950 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:00.950 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=7 2024-03-13 15:55:00.955 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,41d527ac6c6c] t.b.t.CollectAllTenantMiniCompactionTask : Finish to collect all tenants mini compaction info 2024-03-13 15:55:00.955 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2024-03-13 15:55:00.955 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2024-03-13 15:55:00.956 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,e1458fdc5db148cb,1d87c9b6489f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=obcluster1&ObClusterId=1&version=2, client=10.10.10.148, traceId=e1458fdc5db148cb, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getRootServiceInfo(String,Long,String,Long,boolean,Integer), args=obcluster1,1,false,2,] 2024-03-13 15:55:00.958 INFO 1433467 --- [pool-Check obproxy5,f47a761474a84dce,5ce32a29443d] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:55:00.958 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,e1458fdc5db148cb,1d87c9b6489f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=e1458fdc5db148cb, duration=3 ms] 2024-03-13 15:55:00.959 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.948 INFO 1433467 --- [pool-Check obproxy10,f47a761474a84dce,fefdadc2e595] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:55:00.956 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-195,2509f16a1b404931,85a05963a38b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=obcluster&ObClusterId=1&version=2, client=10.10.10.148, traceId=2509f16a1b404931, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getRootServiceInfo(String,Long,String,Long,boolean,Integer), args=obcluster,1,false,2,] 2024-03-13 15:55:00.959 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.959 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.959 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:00.960 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-195,2509f16a1b404931,85a05963a38b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=2509f16a1b404931, duration=5 ms] 2024-03-13 15:55:00.960 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:00.966 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.966 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,41d527ac6c6c] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016735, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.966 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.966 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [rootservice_list] 2024-03-13 15:55:00.966 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:00.969 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [rootservice_list] 2024-03-13 15:55:00.971 INFO 1433467 --- [pool-Check obproxy15,f47a761474a84dce,d2d852e369d7] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:55:00.971 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:00.972 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.972 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:00.972 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.972 INFO 1433467 --- [ForkJoinPool-1386566-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.10.10.148, 2882] 2024-03-13 15:55:00.973 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.perf.sql.internal.SqlParseService : querySqlTextWithParams:1,obcluster,1002,1710288000000000,2000,30000000 2024-03-13 15:55:00.973 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:00.973 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:00.974 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.perf.sql.internal.SqlParseService : parse 0 sqls for tenant:ObTenantEntity{id=4, name=ocp, obTenantId=1002, status=NORMAL} 2024-03-13 15:55:00.977 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.979 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,45b90f3ae487] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016740, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.979 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:00.980 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.980 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.980 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.980 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.10.10.145, 2882] 2024-03-13 15:55:00.981 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,93b21b0dfde5] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:55:00.981 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.981 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:00.982 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.984 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.984 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.10.10.146, 2882] 2024-03-13 15:55:00.982 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.982 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.984 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.986 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:00.986 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:00.987 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:00.988 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,02cc2db91f20] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016742, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:00.988 INFO 1433467 --- [ForkJoinPool-1386566-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port AS inner_port, with_rootserver, UPPER(`status`) AS `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 AS stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 AS start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 AS block_migrate_in_time FROM DBA_OB_SERVERS WHERE svr_ip = ? AND svr_port = ?, args: [10.10.10.147, 2882] 2024-03-13 15:55:00.988 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:55:00.994 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62889/api/v1/log/alarms, params:null 2024-03-13 15:55:01.004 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014385, status:SUCCESS 2024-03-13 15:55:01.004 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014385, status:SUCCESS 2024-03-13 15:55:01.009 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014386, status:SUCCESS 2024-03-13 15:55:01.009 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014386, status:SUCCESS 2024-03-13 15:55:01.013 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014387, status:SUCCESS 2024-03-13 15:55:01.014 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014387, status:SUCCESS 2024-03-13 15:55:01.017 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.o.DiskUsageTimeRemainingOperator : do least Square for obcluster1 2024-03-13 15:55:01.017 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.ocp.diagnosis.model.NeuralNetwork : W:[566.8803467723029], b:[-9.491477515815789E11] 2024-03-13 15:55:01.018 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014388, status:SUCCESS 2024-03-13 15:55:01.019 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014388, status:SUCCESS 2024-03-13 15:55:01.019 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.022 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.023 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014389, status:SUCCESS 2024-03-13 15:55:01.024 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.024 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014389, status:SUCCESS 2024-03-13 15:55:01.026 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.031 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014390, status:SUCCESS 2024-03-13 15:55:01.027 INFO 1433467 --- [pool-scanner-1609793012-1,f47a761474a84dce,ca8d4c673b74] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.031 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.031 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014390, status:SUCCESS 2024-03-13 15:55:01.033 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:55:01.037 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014391, status:SUCCESS 2024-03-13 15:55:01.037 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014391, status:SUCCESS 2024-03-13 15:55:01.038 INFO 1433467 --- [pool-scanner-1609793013-1,f47a761474a84dce,916c0e02707a] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.040 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.043 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:55:01.047 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014392, status:SUCCESS 2024-03-13 15:55:01.049 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014392, status:SUCCESS 2024-03-13 15:55:01.053 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014393, status:SUCCESS 2024-03-13 15:55:01.053 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014393, status:SUCCESS 2024-03-13 15:55:01.058 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.061 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.071 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.073 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.080 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.081 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.083 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:55:01.084 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:55:01.087 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.088 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.090 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [3600] 2024-03-13 15:55:01.103 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.120 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:01.120 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:01.122 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=3600, delayTasks=[] 2024-03-13 15:55:01.131 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.133 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.144 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.146 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.148 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.149 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.151 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:55:01.152 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:55:01.153 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.155 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.156 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [300] 2024-03-13 15:55:01.173 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=300, delayTasks=[] 2024-03-13 15:55:01.188 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.190 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.192 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.193 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.195 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [90] 2024-03-13 15:55:01.224 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=90, timeoutTasks=[] 2024-03-13 15:55:01.231 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.233 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.234 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.238 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.241 INFO 1433467 --- [pool-scanner-1609793048-1,f47a761474a84dce,1645838e796b] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.242 INFO 1433467 --- [pool-scanner-1609793049-1,f47a761474a84dce,ed8bc8e33f83] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=48 2024-03-13 15:55:01.242 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.245 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [60] 2024-03-13 15:55:01.251 INFO 1433467 --- [pool-scanner-1609793051-1,f47a761474a84dce,b67e3a85a04c] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=48 2024-03-13 15:55:01.251 INFO 1433467 --- [pool-scanner-1609793050-1,f47a761474a84dce,289060e945cd] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.255 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.260 INFO 1433467 --- [pool-scanner-1609793053-1,f47a761474a84dce,27a66a065ff6] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.264 INFO 1433467 --- [pool-scanner-1609793052-1,f47a761474a84dce,b8c11b34feb7] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.267 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=60, timeoutTasks=[] 2024-03-13 15:55:01.274 INFO 1433467 --- [pool-scanner-1609793055-1,f47a761474a84dce,6c8e065f171c] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.275 INFO 1433467 --- [pool-scanner-1609793054-1,f47a761474a84dce,cf7bce6364b5] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=6, nodesFilledCount=30 2024-03-13 15:55:01.279 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster1 2024-03-13 15:55:01.279 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.281 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.282 WARN 1433467 --- [pool-scanner-1609793057-1,f47a761474a84dce,15327d2ee038] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = ob_tenant_disk_total_size, seriesId size = 16 2024-03-13 15:55:01.283 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:01.287 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.290 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:55:01.294 INFO 1433467 --- [pool-scanner-1609793057-1,f47a761474a84dce,15327d2ee038] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=16, nodesFilledCount=32 2024-03-13 15:55:01.314 INFO 1433467 --- [pool-scanner-1609793059-1,f47a761474a84dce,9d3be4dac064] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=16, nodesFilledCount=32 2024-03-13 15:55:01.320 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.334 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:55:01.342 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:01.344 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.345 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:01.347 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:01.350 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:55:01.371 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.374 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:55:01.377 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster1 does not enable secondary backup 2024-03-13 15:55:01.379 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster does not enable secondary backup 2024-03-13 15:55:01.383 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackup/], metricValues size:[0]. 2024-03-13 15:55:01.385 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackupocp/], metricValues size:[0]. 2024-03-13 15:55:01.392 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,14961e66e93a] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016732, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:01.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.o.DiskUsageTimeRemainingOperator : do least Square for obcluster 2024-03-13 15:55:01.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.ocp.diagnosis.model.NeuralNetwork : W:[238.19327327213082], b:[-3.98276346274587E11] 2024-03-13 15:55:01.395 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.400 INFO 1433467 --- [pool-scanner-1609793074-1,f47a761474a84dce,52e91b10a0b0] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.408 INFO 1433467 --- [pool-scanner-1609793075-1,f47a761474a84dce,33938c408303] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.410 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.450 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.507 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.509 INFO 1433467 --- [pool-scanner-1609793110-1,f47a761474a84dce,19447b7a4ac5] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.511 INFO 1433467 --- [pool-scanner-1609793111-1,f47a761474a84dce,a83dcbce334a] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=7 2024-03-13 15:55:01.518 INFO 1433467 --- [pool-scanner-1609793112-1,f47a761474a84dce,786e357c943b] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.518 INFO 1433467 --- [pool-scanner-1609793113-1,f47a761474a84dce,eeceb8a8109f] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=7 2024-03-13 15:55:01.521 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.523 INFO 1433467 --- [pool-scanner-1609793114-1,f47a761474a84dce,7f6ee13f7cf3] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.525 INFO 1433467 --- [pool-scanner-1609793115-1,f47a761474a84dce,013900610e7a] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.531 INFO 1433467 --- [pool-scanner-1609793117-1,f47a761474a84dce,130981a6c3db] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.532 INFO 1433467 --- [pool-scanner-1609793116-1,f47a761474a84dce,86ec9e46b791] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=2, nodesFilledCount=4 2024-03-13 15:55:01.534 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.d.DiagnosisFuncUpdateService : update diag func fo cluster:obcluster 2024-03-13 15:55:01.543 INFO 1433467 --- [pool-scanner-1609793119-1,f47a761474a84dce,2df9d374d04e] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=16, nodesFilledCount=32 2024-03-13 15:55:01.559 INFO 1433467 --- [pool-scanner-1609793121-1,f47a761474a84dce,c3b93f9fceca] c.o.o.m.store.RollupMetricDataStore : fetchFromDb, nodesCountFromDb=16, nodesFilledCount=32 2024-03-13 15:55:01.573 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,272acbe53cb7] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016733, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:01.891 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804715,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:02.081 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014383, status:SUCCESS 2024-03-13 15:55:02.082 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014383, status:SUCCESS 2024-03-13 15:55:02.092 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014384, status:SUCCESS 2024-03-13 15:55:02.093 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014384, status:SUCCESS 2024-03-13 15:55:02.146 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,f5b7a114c7b64578,7ceb16e6b8a7] c.o.o.s.c.trace.RequestTracingAspect : API: [POST /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=obcluster1&version=2&ObClusterId=1, client=10.10.10.146, traceId=f5b7a114c7b64578, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.updateRootServiceInfo(String,Long,String,Long,Integer,String), args=obcluster1,1,2,{"ObRegion":"obcluster1","ObCluster":"obcluster1","ObRegionId":1,"ObClusterId":1,"Type":"PRIMARY","timestamp":1710316502144288,"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"ReadonlyRsList":[]},] 2024-03-13 15:55:02.159 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,f5b7a114c7b64578,7ceb16e6b8a7] c.o.ocp.configurl.ConfigUrlService : updateRootServer, clusterName=obcluster1, clusterId=1, affectRows=1 2024-03-13 15:55:02.160 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,f5b7a114c7b64578,7ceb16e6b8a7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [POST /services client=10.10.10.146, traceId=f5b7a114c7b64578, duration=15 ms] 2024-03-13 15:55:03.892 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804738,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:04.120 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:04.121 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:05.892 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804761,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:06.104 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804765,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:06.749 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:06.748755+08:00, lastProcessTime=2024-03-13T15:54:56.745596+08:00 2024-03-13 15:55:06.863 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:06.748755+08:00 2024-03-13 15:55:06.923 INFO 1433467 --- [pool-ocp-schedules-2,79505fbdec8440bf,29721cf315a4] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:07.121 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:07.122 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:07.865 INFO 1433467 --- [pool-15-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : Starting PLAN_CHANGE_DETECTOR, Interval{start=2024-03-13T15:51:07.61, end=2024-03-13T15:53:07.863} 2024-03-13 15:55:07.873 INFO 1433467 --- [pool-15-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR : for [Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null)] 2024-03-13 15:55:07.874 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:55:07.878 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:07.61, end=2024-03-13T15:53:07.863},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:55:07.878 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:55:07.878 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:55:07.887 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:55:07.888 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 1 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:07.61, end=2024-03-13T15:53:07.863},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:55:07.892 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804784,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:07.894 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:55:07.894 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:55:07.948 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107948000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=C3C7525B95B37E68FE1C3D3AC19F8E2B, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.949 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107949000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=B7A34188E00F96CA660B2D39A3968328, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.949 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107949000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.951 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107951000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.951 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107951000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=DD3D1866D187B8E0DC29B621473BC036, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.959 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 233 2024-03-13 15:55:07.959 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.961 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107961000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=C93AE033743B2D4CAB9F14399C625DAF, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.962 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2024-03-13 15:55:07.962 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.965 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107965000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=E204274999A63B8CBD67199F075C5D97, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.966 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 89 2024-03-13 15:55:07.966 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.966 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 200 2024-03-13 15:55:07.966 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.967 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 382 2024-03-13 15:55:07.967 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.967 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230107967000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=ED37973188EACB829E6683C582C7E117, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:07.968 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 274 2024-03-13 15:55:07.968 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.970 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.970 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=B7A34188E00F96CA660B2D39A3968328, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.973 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.973 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=DD3D1866D187B8E0DC29B621473BC036, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=C93AE033743B2D4CAB9F14399C625DAF, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.975 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 258 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.976 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.977 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.977 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.979 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.979 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.981 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.981 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=C3C7525B95B37E68FE1C3D3AC19F8E2B, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.982 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.982 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.984 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.984 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.984 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 245 2024-03-13 15:55:07.984 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:07.985 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.985 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.985 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.985 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=E204274999A63B8CBD67199F075C5D97, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.986 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.986 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=C93AE033743B2D4CAB9F14399C625DAF, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.987 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:07.987 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:07.987 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 6 records,use plan union hash 2024-03-13 15:55:07.987 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.987 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=DD3D1866D187B8E0DC29B621473BC036, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.989 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:55:07.993 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.961, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=C93AE033743B2D4CAB9F14399C625DAF, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:07.993 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:55:07.993 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.993 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.994 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.994 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=B7A34188E00F96CA660B2D39A3968328, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.994 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.994 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=C3C7525B95B37E68FE1C3D3AC19F8E2B, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.995 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.951, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=DD3D1866D187B8E0DC29B621473BC036, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:07.996 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:55:07.996 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.996 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=E204274999A63B8CBD67199F075C5D97, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.997 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.949, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=B7A34188E00F96CA660B2D39A3968328, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:07.997 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:55:07.997 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 1 records,use plan union hash 2024-03-13 15:55:07.998 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:07.998 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=ED37973188EACB829E6683C582C7E117, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:07.999 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.948, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=C3C7525B95B37E68FE1C3D3AC19F8E2B, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:07.999 INFO 1433467 --- [pool-plan_change_detector6,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.951, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:07.999 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 6 records,use plan union hash 2024-03-13 15:55:08.000 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:08.000 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:08.000 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.965, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=E204274999A63B8CBD67199F075C5D97, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:08.006 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 4 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.949, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:08.008 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:08.008 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=ED37973188EACB829E6683C582C7E117, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:08.010 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 4 records,use plan union hash 2024-03-13 15:55:08.013 INFO 1433467 --- [pool-plan_change_detector8,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:07.967, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=2, sqlId=ED37973188EACB829E6683C582C7E117, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:08.013 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:55:08.014 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:07.61, end=2024-03-13T15:53:07.863},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:55:08.014 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:55:08.014 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:55:08.046 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230108046000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=1, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=1, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:55:08.074 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 788 2024-03-13 15:55:08.074 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:55:08.084 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:08.084 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1, obServerId=null, obDbId=201001, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:08.086 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:55:08.087 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:55:08.095 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:55:08.095 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316387863000, obClusterId=1, clusterName=obcluster1, obTenantId=1, obServerId=null, obDbId=201001, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, sqlIdList=null, dynamicSql=false) 2024-03-13 15:55:08.098 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2024-03-13 15:55:08.101 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:55:08.046, end=2024-03-13T15:53:07.863}, clusterId=1, tenantId=1, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:55:08.101 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:55:08.102 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:07.61, end=2024-03-13T15:53:07.863},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:55:08.102 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:55:08.102 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:55:08.103 INFO 1433467 --- [pool-15-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR, Found out 0 dubious sqls 2024-03-13 15:55:09.248 INFO 1433467 --- [pool-ocp-schedules-11,ea026846b4204ce0,d89131997ff0] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2024-03-13 15:55:09.255 INFO 1433467 --- [pool-ocp-schedules-15,c0140cd69d8b4bae,0ddea33cf400] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:55:09.258 INFO 1433467 --- [pool-ocp-schedules-13,c629d149f6af4d69,152121145329] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:55:09.258 INFO 1433467 --- [pool-ocp-schedules-15,c0140cd69d8b4bae,0ddea33cf400] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:55:09.261 INFO 1433467 --- [pool-ocp-schedules-13,c629d149f6af4d69,152121145329] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:55:09.263 INFO 1433467 --- [pool-ocp-schedules-11,ea026846b4204ce0,d89131997ff0] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2024-03-13 15:55:09.264 INFO 1433467 --- [pool-ocp-schedules-6,20c0e83a0d1e4204,fa3100b236cd] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:55:09.266 INFO 1433467 --- [pool-ocp-schedules-6,20c0e83a0d1e4204,fa3100b236cd] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:55:09.267 INFO 1433467 --- [pool-ocp-schedules-12,b3c7547a73e54993,e8846c42b96e] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:55:09.267 INFO 1433467 --- [pool-Check host task for clockdiff25,b3c7547a73e54993,f3124660a200] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff19,b3c7547a73e54993,987ad4c0ca86] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff13,b3c7547a73e54993,17efb8e46ec0] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff11,b3c7547a73e54993,218de21a2ef9] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff26,b3c7547a73e54993,5bcd9b7395f0] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff22,b3c7547a73e54993,865b6468034c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:55:09.268 INFO 1433467 --- [pool-Check host task for clockdiff14,b3c7547a73e54993,8add1b662569] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-ocp-schedules-10,aedc047512554686,2224b5a4936e] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-ocp-schedules-14,416573bd302c485b,b679f86b2c5a] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-ocp-schedules-17,eaff7f3ddae4496f,0e38ee71e4b8] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:55:09.272 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-ocp-schedules-4,6ade802bff964a9d,135af58984c8] c.o.o.s.task.scheduled.ClusterSchedules : Check host version schedule. 2024-03-13 15:55:09.273 INFO 1433467 --- [pool-ocp-schedules-4,6ade802bff964a9d,135af58984c8] o.o.c.t.e.u.DistributedExclusiveExecutor : Empty item list, taskName=Check host task for host version, skip execute 2024-03-13 15:55:09.320 WARN 1433467 --- [pool-Check host task for clockdiff26,b3c7547a73e54993,5bcd9b7395f0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.320 ERROR 1433467 --- [pool-Check host task for clockdiff26,b3c7547a73e54993,5bcd9b7395f0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:55:09.320 WARN 1433467 --- [pool-Check host task for clockdiff26,b3c7547a73e54993,5bcd9b7395f0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.325 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:09.325 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:09.326 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:09.328 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:09.328 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:09.328 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:09.331 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:09.332 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:09.333 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:09.334 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:09.335 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:09.335 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:09.336 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804801,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:09.338 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:09.340 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:09.354 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.354 INFO 1433467 --- [pool-ocp-schedules-3,9c31d9ee96994c77,c945be7d14d3] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:55:09.355 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:55:09.355 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.355 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.357 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:55:09.357 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:55:09.357 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:55:09.357 WARN 1433467 --- [pool-Check host task for clockdiff14,b3c7547a73e54993,8add1b662569] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.357 WARN 1433467 --- [pool-Check host task for clockdiff25,b3c7547a73e54993,f3124660a200] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.358 ERROR 1433467 --- [pool-Check host task for clockdiff14,b3c7547a73e54993,8add1b662569] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:55:09.358 ERROR 1433467 --- [pool-Check host task for clockdiff25,b3c7547a73e54993,f3124660a200] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:55:09.358 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:09.358 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:55:09.358 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:55:09.358 WARN 1433467 --- [pool-Check host task for clockdiff25,b3c7547a73e54993,f3124660a200] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.358 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:55:09.359 WARN 1433467 --- [pool-Check host task for clockdiff19,b3c7547a73e54993,987ad4c0ca86] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.359 ERROR 1433467 --- [pool-Check host task for clockdiff19,b3c7547a73e54993,987ad4c0ca86] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:55:09.359 WARN 1433467 --- [pool-Check host task for clockdiff19,b3c7547a73e54993,987ad4c0ca86] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.358 WARN 1433467 --- [pool-Check host task for clockdiff14,b3c7547a73e54993,8add1b662569] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.363 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:09.363 INFO 1433467 --- [ForkJoinPool-1386568-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.363 INFO 1433467 --- [ForkJoinPool-1386568-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.363 INFO 1433467 --- [ForkJoinPool-1386568-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.364 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:09.364 INFO 1433467 --- [ForkJoinPool-1386568-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:09.364 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:09.364 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:09.366 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:09.366 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:09.366 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:09.367 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:09.367 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:09.367 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:09.367 INFO 1433467 --- [ForkJoinPool-1386568-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:55:09.367 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:09.368 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:09.368 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:09.368 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:09.369 INFO 1433467 --- [ForkJoinPool-1386568-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.369 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:55:09.369 INFO 1433467 --- [ForkJoinPool-1386568-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.369 INFO 1433467 --- [ForkJoinPool-1386568-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.370 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:55:09.371 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:55:09.371 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:55:09.371216023+08:00 2024-03-13 15:55:09.371 INFO 1433467 --- [ForkJoinPool-1386568-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:09.372 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:55:09.371+08:00 2024-03-13 15:55:09.372 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:55:09.373 INFO 1433467 --- [ForkJoinPool-1386568-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:55:09.373 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:55:09.372872640+08:00 2024-03-13 15:55:09.373 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:55:09.373167565+08:00 2024-03-13 15:55:09.374 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:55:09.371+08:00 and status AVAILABLE 2024-03-13 15:55:09.374 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:55:09.373+08:00 2024-03-13 15:55:09.374 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:55:09.373+08:00 2024-03-13 15:55:09.375 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:55:09.374450394+08:00 2024-03-13 15:55:09.376 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:55:09.373+08:00 and status AVAILABLE 2024-03-13 15:55:09.376 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:55:09.377 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:55:09.375+08:00 2024-03-13 15:55:09.377 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:55:09.378 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.378 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.378 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.378 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:09.379 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:55:09.375+08:00 and status AVAILABLE 2024-03-13 15:55:09.379 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:55:09.373+08:00 and status AVAILABLE 2024-03-13 15:55:09.379 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:55:09.379262241+08:00 2024-03-13 15:55:09.380 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:55:09.379+08:00 2024-03-13 15:55:09.380 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:55:09.381 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:55:09.379+08:00 and status AVAILABLE 2024-03-13 15:55:09.381 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:55:09.383 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:55:09.383 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.383 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:55:09.384 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:09.385 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:09.386 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.386 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:55:09.386 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:55:09.386 INFO 1433467 --- [pool-Check host task for agent15,9c31d9ee96994c77,cc08cb1e4c1f] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:55:09.387 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.388 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:55:09.388 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.388 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:09.389 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:09.389 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:09.390 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:55:09.389835713+08:00 2024-03-13 15:55:09.390 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:55:09.390+08:00 2024-03-13 15:55:09.390 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:09.391 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:55:09.390+08:00 and status AVAILABLE 2024-03-13 15:55:09.391 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:55:09.391 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:55:09.391 INFO 1433467 --- [pool-Check host task for agent4,9c31d9ee96994c77,62966d31d200] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent20,9c31d9ee96994c77,cb8b2a36b2a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:55:09.392 INFO 1433467 --- [pool-Check host task for agent30,9c31d9ee96994c77,d71ad584fba1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:55:09.393 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:09.394 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:55:09.395 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:55:09.395 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:55:09.395066471+08:00 2024-03-13 15:55:09.395 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:09.395 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:09.396 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:55:09.395+08:00 2024-03-13 15:55:09.396 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:09.400 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:55:09.395+08:00 and status AVAILABLE 2024-03-13 15:55:09.400 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:09.401 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:09.402 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.402 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.403 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.404 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.406 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:55:09.407 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:55:09.407 INFO 1433467 --- [pool-Check host task for agent28,9c31d9ee96994c77,c9de4c5799a3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:55:09.407 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:55:09.408 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:55:09.411 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:55:09.411 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:55:09.411 INFO 1433467 --- [pool-Check host task for agent16,9c31d9ee96994c77,16a544911d75] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:55:09.422 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.422 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.424 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:09.425 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.425 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:09.429 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.432 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:55:09.433 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:55:09.436 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:55:09.439 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:55:09.439 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:55:09.441 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:09.442 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:55:09.447 WARN 1433467 --- [pool-scanner-1609793253-1,86b62a8260ff47eb,74c2daedead7] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:09.449 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.451 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.451 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.455 WARN 1433467 --- [pool-Check host task for clockdiff22,b3c7547a73e54993,865b6468034c] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.455 ERROR 1433467 --- [pool-Check host task for clockdiff22,b3c7547a73e54993,865b6468034c] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:55:09.455 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:55:09.465 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:55:09.475 WARN 1433467 --- [pool-Check host task for clockdiff13,b3c7547a73e54993,17efb8e46ec0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.469 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:55:09.473 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:55:09.476 ERROR 1433467 --- [pool-Check host task for clockdiff13,b3c7547a73e54993,17efb8e46ec0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:55:09.473 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:55:09.455 WARN 1433467 --- [pool-Check host task for clockdiff11,b3c7547a73e54993,218de21a2ef9] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.475 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:55:09.476 ERROR 1433467 --- [pool-Check host task for clockdiff11,b3c7547a73e54993,218de21a2ef9] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:55:09.476 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:55:09.476 INFO 1433467 --- [pool-Check host task for agent22,9c31d9ee96994c77,8a519e5d0e5d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:55:09.476 WARN 1433467 --- [pool-Check host task for clockdiff11,b3c7547a73e54993,218de21a2ef9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.476 WARN 1433467 --- [pool-Check host task for clockdiff13,b3c7547a73e54993,17efb8e46ec0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.480 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:55:09.482 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:55:09.492 WARN 1433467 --- [pool-Check host task for clockdiff22,b3c7547a73e54993,865b6468034c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:09.509 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:55:09.511 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:55:09.514 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:55:09.516 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:55:09.517 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:55:09.519 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:55:09.525 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:55:09.527 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.527 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.528 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.529 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.531 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.531 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.532 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.532 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.532 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:55:09.535 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:09.536 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.539 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.544 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:55:09.546 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:55:09.550 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:55:09.553 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.554 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.554 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.554 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.555 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:09.556 WARN 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:55:09.557 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.557 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.557 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:55:09.558 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:09.559 INFO 1433467 --- [pool-Check host task for host57,416573bd302c485b,ca6dba2f4fb9] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:55:09.560 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.566 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.566 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:09.566 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:55:09.572 INFO 1433467 --- [pool-Check host task for host14,416573bd302c485b,a80f10388fc8] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:55:09.573 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:55:09.574 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:55:09.575 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.575 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:55:09.577 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:55:09.579 INFO 1433467 --- [pool-Check host task for host38,416573bd302c485b,0a05f73aa7b2] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:55:09.581 INFO 1433467 --- [pool-Check host task for host47,416573bd302c485b,62cfbc729f2b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:55:09.581 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:55:09.582 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.582 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.582 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.583 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.583 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:09.583 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:55:09.586 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:55:09.586 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:55:09.587 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:55:09.588 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:09.589 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:09.593 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:55:09.595 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.602 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:55:09.610 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.615 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:09.635 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:55:09.644 INFO 1433467 --- [pool-Check host task for host18,416573bd302c485b,9b170e3af4bd] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:55:09.653 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:55:09.659 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:55:09.664 INFO 1433467 --- [pool-Check host task for host53,416573bd302c485b,d985b8244a29] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:55:09.667 INFO 1433467 --- [alarm-detect-16,926f03f3a2d6463c,540d6355c939] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000911, data is empty. 2024-03-13 15:55:09.671 INFO 1433467 --- [pool-Check host task for host48,416573bd302c485b,beadfc901925] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:55:09.671 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:55:09.671 INFO 1433467 --- [alarm-detect-16,926f03f3a2d6463c,540d6355c939] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:55:09.678 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.679 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.679 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:09.679 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.680 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:55:09.680 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.680 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.681 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.681 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.681 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:55:09.684 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:55:09.684 INFO 1433467 --- [ForkJoinPool-1386567-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710315940072000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:55:09.718 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.718 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.718 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.718 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.719 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:09.723 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.723 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.724 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:55:09.726 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:09.733 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.733 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:09.742 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.742 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:09.747 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.747 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:09.754 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.754 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:09.759 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.759 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:09.768 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:09.780 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.780 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.780 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.780 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.781 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:09.784 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:55:09.786 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:55:09.788 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:09.789 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:09.808 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:55:09.842 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:55:09.861 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:09.862 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.862 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:09.862 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:09.863 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:55:09.877 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:55:09.878 INFO 1433467 --- [ForkJoinPool-1386567-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:55:09.893 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804810,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:10.123 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:10.124 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:10.400 INFO 1433467 --- [pool-async-queue-offer-interval60-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 60, queue size = 6, ruleSize = 6. 2024-03-13 15:55:10.402 INFO 1433467 --- [pool-async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 17, ruleSize = 17. 2024-03-13 15:55:10.809 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.809 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.813 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.813 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.816 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.816 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.822 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.822 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.917 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:55:10.918 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:10.919 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:55:10.920 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:55:10.940 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:55:10.941 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:55:10.967 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:10.968 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:55:10.969 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:55:10.989 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:55:10.990 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:55:10.991 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:55:10.991 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:55:10.991 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:55:10.991 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:11.000 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:55:11.009 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:55:11.010 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:55:11.010 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:55:11.010 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total 2024-03-13 15:55:11.010 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_route 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_server_process_request_time 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_hit 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_insert 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_update 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_response 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_sequence 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_partition 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_table 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_request 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_request 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_response 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_commit 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_delete 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_update 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_select 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_slow 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_others 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_sequence 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_large 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_prepare_send_request_time 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_success 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_select 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_delete 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_commit 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_others 2024-03-13 15:55:11.014 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_insert 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.017 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.019 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:55:11.019 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:55:11.019 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.019 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.019 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.020 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.023 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.026 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.029 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.032 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.035 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.038 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:55:11.041 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:55:11.893 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804833,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:12.229 INFO 1433467 --- [pool-metric-archive-second-2,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316440, nodeCount=20, affectRows=20 2024-03-13 15:55:12.229 INFO 1433467 --- [pool-metric-archive-second-2,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316440, nodeCount=20, validCount=20 2024-03-13 15:55:12.251 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316440, nodeCount=1000, affectRows=1000 2024-03-13 15:55:12.253 INFO 1433467 --- [pool-metric-archive-second-5,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316440, nodeCount=1000, affectRows=1000 2024-03-13 15:55:12.256 INFO 1433467 --- [pool-metric-archive-second-5,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316440, nodeCount=1000, validCount=1000 2024-03-13 15:55:12.256 INFO 1433467 --- [pool-metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316440, nodeCount=1000, affectRows=1000 2024-03-13 15:55:12.258 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316440, nodeCount=999, validCount=999 2024-03-13 15:55:12.259 INFO 1433467 --- [pool-metric-archive-second-7,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316440, nodeCount=1000, validCount=1000 2024-03-13 15:55:12.734 INFO 1433467 --- [pool-ocp-schedules-1,83737baf3ee94e07,7d6211d96660] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:55:12.753 INFO 1433467 --- [pool-ocp-schedules-1,83737baf3ee94e07,7d6211d96660] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:12.753 INFO 1433467 --- [pool-ocp-schedules-1,83737baf3ee94e07,7d6211d96660] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:13.124 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:13.128 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:13.894 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804857,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:15.102 INFO 1433467 --- [pool-prometheus-second-schedule-1,53b9a980251443db,f7667bacc563] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=23, collectTaskCount=23 2024-03-13 15:55:15.781 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,4c9ef00321e845a1,4da20a8a5e7f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.148, traceId=4c9ef00321e845a1, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:55:15.793 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,4c9ef00321e845a1,4da20a8a5e7f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=4c9ef00321e845a1, duration=12 ms] 2024-03-13 15:55:15.894 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804880,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:16.104 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804884,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:16.128 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:16.129 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:16.180 INFO 1433467 --- [pool-ocp-schedules-19,dbd2dafdcd0f497c,5f20b3993112] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":12075,"minuteWriteQueue":0,"minuteReadCacheSize":12328,"secondWriteCacheSize":2488,"seriesIdKeySize":67930,"secondWriteQueue":0,"minuteWriteCacheSize":12393} 2024-03-13 15:55:16.180 INFO 1433467 --- [pool-ocp-schedules-19,dbd2dafdcd0f497c,5f20b3993112] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{go_memory_classes_heap_stacks_bytes=15, ob_sysstat_40012=19, ob_server_resource_memory_assigned_bytes=13, ob_sysstat_40011=19, ob_sysstat_40010=19, ob_unit_num=13, ob_database_disk_required_size=35, go_memstats_mspan_sys_bytes=15, go_memstats_stack_sys_bytes=15, ob_tenant_request_queue_total_size=18, jvm_gc_memory_allocated_bytes_total=3, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=18, ob_sysstat_40019=19, observer_fd_count=13, ob_sysstat_40018=19, processor_process_metrics_total=21, jvm_gc_max_data_size_bytes=3, go_memstats_frees_total=15, sql_audit_input_collect_data_jumped_total=8, host_available=64, mysql_output_write_sql_failed_count=17, ob_database_disk_data_size=34, ob_system_event_detail_total_timeouts=763, ob_tenant_memstore_freeze_cnt=14, ob_latch_sleep_total=312, ob_tenant_unit_min_memory=25, ob_sysstat_60087=19, ob_memstore_total_bytes=20, metric_cardinality_count=12, process_resident_memory_bytes=15, jvm_threads_states_threads=8, ob_cache_size_bytes=34, ob_tenant_unit_max_memory=25, input_collect_metrics_total=26, ob_sysstat_60083=19, ob_sysstat_50039=19, ob_sysstat_50038=19, node_disk_io_time_weighted_seconds_total=33, process_open_fds=15, ob_sysstat_50037=19, process_memory_rss_bytes=33, ob_server_resource_cpu_assigned_percent=13, ob_sysstat_50030=19, go_memory_classes_profiling_buckets_bytes=15, go_memstats_heap_inuse_bytes=15, ob_sysstat_40118=19, ob_sysstat_40117=19, ob_sysstat_40116=19, node_memory_memfree_bytes=26, ob_sysstat_40005=19, process_exists=43, ob_sysstat_40004=19, ob_sysstat_40003=19, ob_sysstat_40002=19, ob_sysstat_40001=19, go_gc_duration_seconds=20, go_memory_classes_metadata_mspan_inuse_bytes=15, ob_server_resource_disk_bytes=13, ob_sysstat_40000=19, odp_current_session=26, process_thread_count=33, ob_sysstat_50040=19, ob_sysstat_40009=19, ob_sysstat_40008=19, ob_disk_free_bytes=13, node_filesystem_avail_bytes=72, ob_sysstat_40007=19, ob_sysstat_40006=19, ob_backup_storage_capacity_bytes=650, ob_unit_config_max_iops=21, ob_plan_cache_memory_bytes=18, node_network_transmit_errs_total=28, go_gc_duration_seconds_count=15, monagent_oceanbase_plan_explain_size=4, ob_sysstat_50011=19, ob_sysstat_50010=19, node_disk_read_bytes_total=33, ob_compaction_is_suspended=19, mysql_output_write_sql_milliseconds_summary_count=17, go_memstats_lookups_total=15, mysql_output_write_sql_count=17, process_cpu_percent=33, ob_sysstat_60057=19, host_agent_process_status=18, ob_server_resource_cpu=13, ob_tenant_server_data_index_row_count=20, go_memstats_alloc_bytes=15, go_sched_gomaxprocs_threads=15, node_cpu_seconds_total_native=38, ob_sysstat_140002=19, ob_sysstat_140005=19, ob_sysstat_140006=19, ob_sysstat_140003=19, ob_tenant_resource_min_memory=18, ob_memstore_freeze_times=20, ob_sysstat_50022=19, jvm_gc_pause_seconds_count=8, ob_sysstat_50021=19, jvm_gc_overhead_percent=3, go_memory_classes_heap_unused_bytes=15, ob_cluster_status_check=13, jvm_memory_committed_bytes=10, node_net_bandwidth_bps=29, monagent_oceanbase_db_pool_idle=4, ob_tenant_server_data_index_data_size=20, go_gc_heap_allocs_by_size_bytes_total_count=15, go_memstats_heap_idle_bytes=15, go_gc_cycles_forced_gc_cycles_total=15, go_gc_pauses_seconds_total_bucket=26, ob_server_resource_memory_bytes=13, ob_tenant_assigned_mem_assigned=18, ob_tenant_memstore_minor_freeze_times=14, ob_tenant_server_required_size=22, ob_sysstat_60031=19, process_fd_count=33, ob_tenant_unit_max_cpu=25, ob_latch_wait_seconds_total=312, go_memory_classes_metadata_mcache_free_bytes=15, node_memory_buffers_bytes=26, ob_sysstat_50009=19, go_memstats_heap_released_bytes=15, node_network_receive_bytes_total=28, ob_sysstat_50005=19, ob_sysstat_50008=19, ob_sysstat_50001=19, ob_sysstat_50004=19, exposer_request_latencies_count=6, ob_sysstat_50000=19, ob_session_active_num=18, mysql_output_context_timeout_discard_metrics_count=9, go_memory_classes_metadata_mcache_inuse_bytes=15, node_disk_read_time_seconds_total=33, ob_sysstat_60021=19, ob_sysstat_60020=19, ob_sysstat_60024=19, ob_compaction_frozen_version=19, ob_sysstat_60023=19, metric_count=15, ob_sysstat_60022=19, obproxysys_connectable=1556, ob_tenant_assigned_cpu_assigned=18, mysql_output_write_sql_retry_count=17, ob_tenant_unit_min_cpu=25, ob_unit_config_max_cpu=21, http_request_milliseconds_summary_count=75, monagent_oceanbase_audit_timeinfo_cache_size=4, http_request_milliseconds_summary=78, ob_latch_immediate_miss_total=312, ob_latch_spin_get_total=312, ob_log_level_count=17, monagent_pipeline_report_metrics_total=16, exposer_transferred_bytes_total=6, ob_tenant_context_memory_count=841, node_disk_reads_completed_total=33, ob_sysstat_60019=19, ob_sysstat_60003=19, ob_sysstat_60002=19, ob_sysstat_60001=19, ob_sysstat_60000=19, ob_latch_miss_total=312, go_sched_latencies_seconds_count=15, ocp_meta_db_health=5, ob_disk_total_bytes=13, go_memstats_heap_objects=15, go_memstats_mcache_inuse_bytes=15, ob_sysstat_60005=19, ob_sysstat_60004=19, go_memory_classes_other_bytes=15, ob_memstore_active_bytes=20, go_memstats_alloc_bytes_total=15, ob_tenant_server_partition_replica_count=21, process_count=34, ob_system_event_detail_total_waits=763, odp_used_connections=25, backup_storage_capacity_used_percent=8, go_gc_heap_frees_objects_total=15, go_memory_classes_metadata_mspan_free_bytes=15, ob_disk_log_used_bytes=13, ob_compaction_is_error=19, go_gc_limiter_last_enabled_gc_cycle=15, monagent_pipeline_execute_total=24, node_filesystem_files=86, process_start_time_seconds=15, go_memstats_heap_alloc_bytes=15, ob_tenant_resource_max_cpu=18, ntp_service_available=26, go_memstats_mspan_inuse_bytes=15, mysql_output_write_sql_milliseconds_summary=20, go_gc_heap_frees_by_size_bytes_total_count=15, monagent_pipeline_execute_seconds_total=24, ob_sysstat_80001=19, go_memstats_sys_bytes=15, go_memory_classes_heap_free_bytes=15, system_load_average_1m=3, monagent_oceanbase_audit_stat_cache_size=4, ob_tenant_memory_used_bytes=22, obproxy_server_max_connections=7, ob_sysstat_80002=19, go_memstats_gc_cpu_fraction=15, ob_sysstat_80003=19, monagent_oceanbase_audit_cache_size=4, ob_tenant_context_memory_used_bytes=841, ob_sysstat_80007=19, node_disk_writes_completed_total=33, ob_sysstat_80008=19, ob_sysstat_80009=19, go_sched_goroutines_goroutines=15, go_memory_classes_metadata_other_bytes=15, go_gc_cycles_total_gc_cycles_total=15, system_cpu_usage=3, process_cpu_usage=3, node_disk_written_bytes_total=33, monagent_oceanbase_ash_cache_size=4, go_goroutines=15, ob_sysstat_80023=19, exposer_request_latencies_sum=6, ob_unit_config_min_cpu=21, process_virtual_memory_max_bytes=15, go_gc_heap_frees_bytes_total=15, ob_tenant500_memory_hold_bytes=13, exposer_scrapes_total=6, go_gc_heap_allocs_by_size_bytes_total_bucket=27, ob_sysstat_80025=19, ob_sysstat_80026=19, oceanbase_connectivity=15, node_memory_sreclaimable_bytes=26, go_sched_latencies_seconds_sum=15, tcp_retrans=25, exporter_export_metrics_bytes_total=26, node_filesystem_size_bytes=72, http_server_requests_seconds_count=203, ob_compaction_frozen_time=19, monagent_input_sql_duration_seconds_sum=55, log_stream_degraded_count=18, ob_plan_cache_hit_total=18, node_filesystem_readonly=72, ob_sysstat_80040=19, ob_sysstat_80041=19, io_await=52, go_gc_heap_allocs_objects_total=15, go_memstats_mallocs_total=15, ob_unit_config_max_memory_bytes=21, go_gc_stack_starting_size_bytes=15, io_util=52, ob_sysstat_200002=19, ob_sysstat_200001=19, ob_tenant_disk_v4_data_size=18, go_sched_latencies_seconds_bucket=26, ob_tenant_assigned_cpu_total=18, ob_tenant_server_available_disk_free=19, node_disk_write_time_seconds_total=33, cpu_count=25, output_write_metrics_total=9, ob_cluster_sync=15, go_gc_heap_objects_objects=15, odp_transaction_total=16, ob_system_event_detail_time_waited_us=763, process_max_fds=15, http_status_count=20, ob_sysstat_80057=19, ocp_alarm_detect_duration_ms_count=115, go_gc_heap_allocs_by_size_bytes_total_sum=15, go_memstats_mcache_sys_bytes=15, ob_tenant_disk_v4_log_size=18, go_cgo_go_to_c_calls_calls_total=15, ocp_alarm_detect_duration_ms_sum=115, jvm_threads_live_threads=3, node_memory_cached_bytes=26, ob_table_num=16, ob_sysstat_10000=19, ob_sysstat_10006=19, ob_sysstat_10005=19, ob_sysstat_10002=19, ob_sysstat_10001=19, go_memstats_last_gc_time_seconds=15, ob_sysstat_10004=19, ob_sysstat_10003=19, monitor_exporter_avaliable=62, log_tailer_count=15, ob_server_num=25, ob_session_all_num=18, ob_system_event_time_waited=24, ob_tenant_resource_min_cpu=18, monagent_plugin_execute_seconds_total=39, go_gc_cycles_automatic_gc_cycles_total=15, go_gc_heap_allocs_bytes_total=15, go_memstats_gc_sys_bytes=15, ob_latch_get_total=312, go_info=16, log_tailer_processed_file_count=21, node_load15=24, ob_disk_log_assigned_bytes=13, go_memory_classes_heap_released_bytes=15, jvm_memory_max_bytes=10, ob_unit_config_min_iops=21, ob_sysstat_30080=19, ob_sysstat_30082=19, ob_sysstat_30081=19, ob_sysstat_30084=19, ob_sysstat_30083=19, ob_sysstat_30086=19, ob_sysstat_30085=19, ob_tenant_log_disk_total_bytes=18, ob_memstore_freeze_trigger_bytes=20, ob_waitevent_wait_seconds_total=18, ob_server_stopped_duration_seconds=15, go_memstats_next_gc_bytes=15, node_filesystem_files_free=86, ob_memtable_max_snapshot_duration_seconds=18, monagent_oceanbase_audit_unique_key_cache_size=4, process_virtual_memory_bytes=15, ob_sysstat_20002=19, net_connectivity=15, ob_sysstat_20001=19, monagent_input_sql_duration_seconds_bucket=65, ob_latch_immediate_get_total=312, node_cpu_seconds_total=32, ob_sysstat_20000=19, ob_index_error_num=13, ob_tenant_server_available_disk_total=19, ob_tenant_memory_hold_bytes=22, ob_sysstat_30066=19, go_memstats_buck_hash_sys_bytes=15, mysql_output_write_sql_milliseconds_summary_sum=17, ob_tenant_disk_total_size=18, odp_sql_request_total=53, mysql_output_write_task_size=17, exporter_export_metrics_total=26, ob_disk_log_total_bytes=13, system_cpu_count=3, ob_sysstat_130004=19, process_uptime_seconds=33, node_ntp_offset_seconds=25, go_memstats_heap_sys_bytes=15, ob_sysstat_140012=19, ob_sysstat_140013=19, jvm_gc_pause_seconds_max=8, ob_tenant_request_queue_large_size=18, ob_sysstat_130000=19, go_memstats_other_sys_bytes=15, ob_sysstat_130001=19, jvm_memory_used_bytes=10, ob_sysstat_130002=19, go_threads=15, http_request_milliseconds_summary_sum=75, http_server_requests_seconds_sum=203, ob_tenant_assigned_mem_total=18, node_memory_memtotal_bytes=26, node_network_transmit_drop_total=28, go_gc_pauses_seconds_total_count=15, exposer_request_latencies=9, ob_waitevent_wait_total=18, go_gc_heap_goal_bytes=15, ob_tenant_log_disk_used_bytes=18, jvm_gc_memory_promoted_bytes_total=3, odp_sql_cost_total=51, ob_sysstat_30011=19, ob_sysstat_30010=19, ob_sysstat_30013=19, ob_sysstat_30012=19, ob_tenant500_memory_used_bytes=13, node_network_transmit_bytes_total=28, node_network_receive_drop_total=28, go_gc_heap_tiny_allocs_objects_total=15, go_gc_duration_seconds_sum=15, ob_server_resource_memory_assigned_percent=13, jvm_gc_pause_seconds_sum=8, go_memory_classes_heap_objects_bytes=15, node_network_receive_errs_total=28, observer_ulimit_max_fd_count=13, ob_plan_cache_access_total=18, ob_tenant_server_row_count=22, monagent_oceanbase_plan_cache_size=4, ob_tenant_server_data_size=22, monagent_oceanbase_audit_sample_cache_size=4, go_gc_heap_frees_by_size_bytes_total_sum=15, ob_sysstat_40030=19, node_load1=24, process_cpu_seconds_total=15, ob_sysstat_120001=19, go_memory_classes_total_bytes=15, ob_sysstat_120002=19, ob_sysstat_120000=19, ob_tenant_resource_max_memory=18, ob_sysstat_120009=19, go_memstats_stack_inuse_bytes=15, ob_sysstat_120008=19, node_load5=24, ob_sysstat_120005=19, go_memory_classes_os_stacks_bytes=15, ob_sysstat_120006=19, jvm_threads_peak_threads=3, monagent_input_sql_duration_seconds_count=55, ob_tenant_context_memory_hold_bytes=841, obproxy_memory_limit_bytes=7, ob_compaction_last_version=19, ob_sysstat_30000=19, ob_server_resource_cpu_assigned=13, ob_sysstat_30002=19, ob_sysstat_30001=19, monagent_oceanbase_plan_timeinfo_cache_size=4, monagent_plugin_execute_total=39, ob_sysstat_30006=19, ob_sysstat_30005=19, ob_sysstat_30008=19, ob_sysstat_30007=19, ob_sysstat_30009=19, go_gc_heap_frees_by_size_bytes_total_bucket=27, obproxy_client_max_connections=7, ob_compaction_current_timestamp=19, ob_clog_max_sync_delay_seconds=20, go_gc_pauses_seconds_total_sum=15, ob_compaction_start_time=19} 2024-03-13 15:55:16.753 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:16.753034+08:00, lastProcessTime=2024-03-13T15:55:06.748755+08:00 2024-03-13 15:55:16.854 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:16.753034+08:00 2024-03-13 15:55:16.930 INFO 1433467 --- [pool-ocp-schedules-10,7b01f3efc8d54b45,38cc5dc370ee] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:17.894 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804904,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:19.129 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:19.130 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:19.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804920,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:19.286 INFO 1433467 --- [alarm-detect-11,2fb6046976de4595,d6b6e0bc8797] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000911, data is empty. 2024-03-13 15:55:19.287 INFO 1433467 --- [alarm-detect-26,d624df6835cc41b9,8b639edaa17c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000911, data is empty. 2024-03-13 15:55:19.288 INFO 1433467 --- [alarm-detect-17,d79c77ff5c2348fe,01b61a0fc032] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000911, data is empty. 2024-03-13 15:55:19.291 INFO 1433467 --- [alarm-detect-26,d624df6835cc41b9,8b639edaa17c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:55:19.291 INFO 1433467 --- [alarm-detect-11,2fb6046976de4595,d6b6e0bc8797] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:55:19.292 INFO 1433467 --- [alarm-detect-17,d79c77ff5c2348fe,01b61a0fc032] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:55:19.314 WARN 1433467 --- [pool-scanner-1609794904-1,7b80a82cb26849f1,ff798b9bb763] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:19.436 INFO 1433467 --- [alarm-detect-2,ea6085b798594005,cdda1754a253] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316485 ~ 1710316509 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316500=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:55:19.436 INFO 1433467 --- [alarm-detect-2,ea6085b798594005,cdda1754a253] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316485 ~ 1710316509 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316500=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:55:19.436 INFO 1433467 --- [alarm-detect-2,ea6085b798594005,cdda1754a253] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316485 ~ 1710316509 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316500=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:55:19.436 INFO 1433467 --- [alarm-detect-2,ea6085b798594005,cdda1754a253] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316485 ~ 1710316509 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316500=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:55:19.436 INFO 1433467 --- [alarm-detect-2,ea6085b798594005,cdda1754a253] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316485 ~ 1710316509 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316500=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:55:19.821 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,3dcf9f6bf82041f6,57efce041dde] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.143, traceId=3dcf9f6bf82041f6, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:55:19.827 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,3dcf9f6bf82041f6,57efce041dde] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.143, traceId=3dcf9f6bf82041f6, duration=7 ms] 2024-03-13 15:55:19.895 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804927,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:20.232 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:21.0, nextRun=2024-03-13 15:55:20.0, currentDate=Wed Mar 13 15:55:20 CST 2024 2024-03-13 15:55:20.233 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999996, name=Collect all cluster compaction info, templateName=CollectClusterCompactionSchedule, scheduleRule=20 * * * * *, lastRunTime=2024-03-13 15:54:21.0, nextRunTime=2024-03-13 15:55:20.0} 2024-03-13 15:55:20.239 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all cluster compaction info, nodeCount=1, taskCount=1 2024-03-13 15:55:20.239 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:20.239 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all cluster compaction info, taskDefinitionId=999996, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:21.108 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016743, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:21.127 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,901bdd6f65b3] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016743, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014394, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:21.102+08:00, sub_task_instance_name=Collect compaction for all clusters, sub_task_instance_id=10016743}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:21.128 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,901bdd6f65b3] c.o.o.s.t.b.c.AbstractAllClusterTask : Collect compaction for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:55:21.129 INFO 1433467 --- [ForkJoinPool-1386573-worker-1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:55:21.129 INFO 1433467 --- [ForkJoinPool-1386573-worker-2,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:55:21.133 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,901bdd6f65b3] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016743, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:21.462 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014394, status:SUCCESS 2024-03-13 15:55:21.467 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014394, status:SUCCESS 2024-03-13 15:55:21.895 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804950,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:22.131 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:22.131 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:23.896 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804973,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:25.132 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:25.133 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:25.218 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,d4ba6b730a97413d,15f90ecc11c6] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.144, traceId=d4ba6b730a97413d, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:55:25.224 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,d4ba6b730a97413d,15f90ecc11c6] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.144, traceId=d4ba6b730a97413d, duration=7 ms] 2024-03-13 15:55:25.896 INFO 1433467 --- [SimpleAsyncTaskExecutor-89804996,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:26.104 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805000,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:26.212 INFO 1433467 --- [pool-Check obproxy7,01c0f21c4d4e4400,9e8f57e710c4] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:55:26.214 INFO 1433467 --- [pool-Check obproxy2,01c0f21c4d4e4400,4931be4f4582] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:55:26.216 INFO 1433467 --- [pool-Check obproxy9,01c0f21c4d4e4400,05c82419f13c] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:55:26.218 INFO 1433467 --- [pool-Check obproxy3,01c0f21c4d4e4400,79eb8c70802e] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:55:26.749 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:26.748088+08:00, lastProcessTime=2024-03-13T15:55:16.753034+08:00 2024-03-13 15:55:26.843 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:26.748088+08:00 2024-03-13 15:55:26.936 INFO 1433467 --- [pool-ocp-schedules-7,f363e302f68542f8,36f43483a4a2] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:27.897 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805020,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:28.133 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:28.133 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:28.794 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.794 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.794 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.795 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:55:28.801 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.801 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.803 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:55:28.804 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule start 2024-03-13 15:55:28.854 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING} 2024-03-13 15:55:28.875 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=2, name=obcluster, obClusterId=1, status=RUNNING} 2024-03-13 15:55:28.882 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster1:1-3) 2024-03-13 15:55:28.882 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster:1-1) 2024-03-13 15:55:28.882 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster1:1-1) 2024-03-13 15:55:28.882 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster1:1-2) 2024-03-13 15:55:28.888 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.888 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.888 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.889 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.889 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.889 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.889 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:55:28.889 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.890 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.890 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.890 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:55:28.890 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2024-03-13 15:55:28.891 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2024-03-13 15:55:28.892 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:28.892 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:28.892 INFO 1433467 --- [sql-rollup-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2024-03-13 15:55:28.935 INFO 1433467 --- [sql-rollup-worker-2,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=obcluster1:1, server=2, level=0->1, c=0) - source=2024-03-13T15:53:02, start=2024-03-13T15:50:02, end=2024-03-13T15:52:32, updates=114, granule=120000ms, delay=180000ms 2024-03-13 15:55:28.942 INFO 1433467 --- [sql-rollup-worker-2,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=2, level=0->1, c=1) - not enough cached interval (source=2024-03-13T15:53:02, target=2024-03-13T15:52:32, granule=120000ms, delay=30000ms) 2024-03-13 15:55:28.943 INFO 1433467 --- [sql-rollup-worker-2,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=2, level=1->2, c=0) - not enough cached interval (source=2024-03-13T15:52:32, target=2024-03-13T15:40:32, granule=600000ms, delay=720000ms) 2024-03-13 15:55:28.944 INFO 1433467 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=obcluster1:1, server=3, level=0->1, c=0) - source=2024-03-13T15:53:00, start=2024-03-13T15:50:30, end=2024-03-13T15:53:00, updates=77, granule=120000ms, delay=150000ms 2024-03-13 15:55:28.944 INFO 1433467 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=obcluster1:1, server=1, level=0->1, c=0) - source=2024-03-13T15:53:02, start=2024-03-13T15:50:02, end=2024-03-13T15:52:32, updates=244, granule=120000ms, delay=180000ms 2024-03-13 15:55:28.946 INFO 1433467 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=3, level=0->1, c=1) - not enough cached interval (source=2024-03-13T15:53:00, target=2024-03-13T15:53:00, granule=120000ms, delay=0ms) 2024-03-13 15:55:28.946 INFO 1433467 --- [sql-rollup-worker-1,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=3, level=1->2, c=0) - not enough cached interval (source=2024-03-13T15:53:00, target=2024-03-13T15:46:00, granule=600000ms, delay=420000ms) 2024-03-13 15:55:28.947 INFO 1433467 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=1, level=0->1, c=1) - not enough cached interval (source=2024-03-13T15:53:02, target=2024-03-13T15:52:32, granule=120000ms, delay=30000ms) 2024-03-13 15:55:28.947 INFO 1433467 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=1, level=1->2, c=0) - not enough cached interval (source=2024-03-13T15:52:32, target=2024-03-13T15:43:02, granule=600000ms, delay=570000ms) 2024-03-13 15:55:28.947 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule end, elapsed=143ms, period=120000ms 2024-03-13 15:55:29.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805037,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:29.330 WARN 1433467 --- [pool-scanner-1609795526-1,2085cc503c8248fe,46e54f9a6018] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:29.417 INFO 1433467 --- [alarm-detect-4,c67c05e4702b43fb,9835c696be4f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:55:29.420 INFO 1433467 --- [alarm-detect-4,c67c05e4702b43fb,9835c696be4f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:55:29.507 INFO 1433467 --- [alarm-detect-25,664659f4441d4e98,5859d7e7b870] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316500 ~ 1710316519 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316515=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:55:29.507 INFO 1433467 --- [alarm-detect-25,664659f4441d4e98,5859d7e7b870] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316500 ~ 1710316519 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316515=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:55:29.507 INFO 1433467 --- [alarm-detect-25,664659f4441d4e98,5859d7e7b870] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316500 ~ 1710316519 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316515=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:55:29.507 INFO 1433467 --- [alarm-detect-25,664659f4441d4e98,5859d7e7b870] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316500 ~ 1710316519 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316515=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:55:29.897 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805044,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:30.569 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:31.0, nextRun=2024-03-13 15:55:30.0, currentDate=Wed Mar 13 15:55:30 CST 2024 2024-03-13 15:55:30.569 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999990, name=Sync tenant information, templateName=TenantSyncSchedule, scheduleRule=30 0/1 * * * ?, lastRunTime=2024-03-13 15:54:31.0, nextRunTime=2024-03-13 15:55:30.0} 2024-03-13 15:55:30.574 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Sync tenant information, nodeCount=1, taskCount=2 2024-03-13 15:55:30.574 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:30.574 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:30.574 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Sync tenant information, taskDefinitionId=999990, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:30.593 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:01.0, nextRun=2024-03-13 15:55:30.0, currentDate=Wed Mar 13 15:55:30 CST 2024 2024-03-13 15:55:30.593 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:55:01.0, nextRunTime=2024-03-13 15:55:30.0} 2024-03-13 15:55:30.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:55:30.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:55:30.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:55:31.134 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:31.135 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:31.214 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016744, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:31.226 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016745, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:31.231 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016744, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014395, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:31.212+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016744}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:55:31.237 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016746, operation:EXECUTE, state: RUNNING 2024-03-13 15:55:31.256 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016745, context=Context{parallelIdx=1, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014395, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:31.224+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016745}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:55:31.257 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,1575a0611d9d] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016746, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014396, task_operation=execute, latest_execution_start_time=2024-03-13T15:55:31.235+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016746}, listMap={}}, executor=10.10.10.148 2024-03-13 15:55:31.268 INFO 1433467 --- [pool-Check obproxy6,f47a761474a84dce,e3e5ee02674f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:55:31.270 INFO 1433467 --- [pool-Check obproxy11,f47a761474a84dce,f74dd2770a1e] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:55:31.270 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,03080f25cdd9] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:55:31.270 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,ea3babfaf05a] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:55:31.279 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.279 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:55:31.279 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.279 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.280 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:55:31.290 INFO 1433467 --- [pool-Check obproxy6,f47a761474a84dce,e3e5ee02674f] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:55:31.293 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,03080f25cdd9] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:55:31.293 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.294 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.294 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.295 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.295 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:55:31.295 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.295 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.295 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:55:31.297 INFO 1433467 --- [pool-Check obproxy11,f47a761474a84dce,f74dd2770a1e] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:55:31.298 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,ea3babfaf05a] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:55:31.300 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.300 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.301 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.303 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.303 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.304 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.307 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,1575a0611d9d] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016746, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:31.307 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.307 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.307 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.312 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.312 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.312 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.313 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.313 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.313 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.318 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.318 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.318 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.321 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.321 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.321 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.343 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.344 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.344 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.344 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.345 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:55:31.354 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.354 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.355 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.355 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.355 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:55:31.357 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.357 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.358 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.358 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.358 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:55:31.366 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.367 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.367 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.367 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.367 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.367 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:55:31.368 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.368 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.368 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.368 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:55:31.377 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.378 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.378 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.378 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.378 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:55:31.381 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.382 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.382 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.382 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.382 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:55:31.387 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.387 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.387 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.387 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.388 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:55:31.392 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:55:31.393 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.394 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=2 2024-03-13 15:55:31.394 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:55:31.398 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:55:31.404 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.404 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.404 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.404 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.405 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:55:31.412 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.413 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.414 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.414 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.414 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.414 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.415 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.415 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.415 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.415 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:55:31.417 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2024-03-13 15:55:31.418 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:55:31.428 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=4, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:55:31.430 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:31.431 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.431 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=4, userWhitelist=% 2024-03-13 15:55:31.432 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.432 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.432 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.433 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:31.433 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.433 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:55:31.439 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:55:31.440 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=3, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=3, userWhitelist=% 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=2 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:31.441 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:55:31.444 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,530c74c24c9d] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016745, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:31.447 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:55:31.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2024-03-13 15:55:31.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2024-03-13 15:55:31.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:55:31.455 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a3e97eb835b0] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016744, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:55:31.653 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014395, status:SUCCESS 2024-03-13 15:55:31.653 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014395, status:SUCCESS 2024-03-13 15:55:31.661 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014396, status:SUCCESS 2024-03-13 15:55:31.661 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014396, status:SUCCESS 2024-03-13 15:55:31.897 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805066,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:32.755 INFO 1433467 --- [pool-ocp-schedules-12,0f67cf35bc654167,402666aacb17] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:55:33.174 INFO 1433467 --- [pool-ocp-schedules-12,0f67cf35bc654167,402666aacb17] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:33.174 INFO 1433467 --- [pool-ocp-schedules-12,0f67cf35bc654167,402666aacb17] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:33.233 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,52ea19a335f549a5,870ff763446a] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.142, traceId=52ea19a335f549a5, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:55:33.238 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,52ea19a335f549a5,870ff763446a] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.142, traceId=52ea19a335f549a5, duration=6 ms] 2024-03-13 15:55:33.898 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805089,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:34.135 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:34.136 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:35.898 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805112,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:36.105 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805116,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:36.748 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:36.747607+08:00, lastProcessTime=2024-03-13T15:55:26.748088+08:00 2024-03-13 15:55:36.881 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:36.747607+08:00 2024-03-13 15:55:36.944 INFO 1433467 --- [pool-ocp-schedules-4,2e7cf03823f94984,a4ba3a72229b] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:37.136 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:37.137 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:37.899 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805136,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:39.256 INFO 1433467 --- [pool-ocp-schedules-4,6ce075921d0c4f07,84c86b540b65] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:55:39.257 INFO 1433467 --- [pool-ocp-schedules-11,313f4e553fa24619,9b7f1c4a0fdb] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:55:39.259 INFO 1433467 --- [pool-ocp-schedules-14,6a7971e20b4747ed,f3289b9cf23e] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:55:39.260 INFO 1433467 --- [pool-ocp-schedules-11,313f4e553fa24619,9b7f1c4a0fdb] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:55:39.261 INFO 1433467 --- [pool-ocp-schedules-14,6a7971e20b4747ed,f3289b9cf23e] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:55:39.261 INFO 1433467 --- [pool-ocp-schedules-4,6ce075921d0c4f07,84c86b540b65] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:55:39.283 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805153,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:39.288 INFO 1433467 --- [pool-ocp-schedules-7,0e3c9d5b34fc485c,7274bcfb1670] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:55:39.290 INFO 1433467 --- [pool-ocp-schedules-2,23c294bd7f1040e7,3ebc107ad1df] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:55:39.291 INFO 1433467 --- [alarm-detect-30,72245d80b10e453a,30eb70939b49] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:55:39.293 INFO 1433467 --- [alarm-detect-17,407e54aa05d5406b,207efd599445] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:55:39.293 INFO 1433467 --- [alarm-detect-11,530eaef250264b2c,a779a6fc76e7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:55:39.299 INFO 1433467 --- [alarm-detect-17,407e54aa05d5406b,207efd599445] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:55:39.300 INFO 1433467 --- [alarm-detect-30,72245d80b10e453a,30eb70939b49] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:55:39.301 INFO 1433467 --- [alarm-detect-11,530eaef250264b2c,a779a6fc76e7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:55:39.306 INFO 1433467 --- [pool-ocp-schedules-8,c8a7656cdcc94f9b,b869de4ee501] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:55:39.307 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:55:39.309 INFO 1433467 --- [pool-ocp-schedules-17,c2b88db545634026,2552a6cfdfcb] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff8,c2b88db545634026,c485b623ec2c] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff3,c2b88db545634026,d1bb7f70dd99] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff29,c2b88db545634026,2d127af644d8] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff1,c2b88db545634026,544fe2e944c0] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff2,c2b88db545634026,b96c463cc429] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for clockdiff24,c2b88db545634026,6e6a70bc9526] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:55:39.310 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:39.311 INFO 1433467 --- [pool-Check host task for clockdiff6,c2b88db545634026,147b8b80b94f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:55:39.312 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:39.312 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:39.312 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:39.312 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:39.313 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:39.313 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:39.313 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:39.314 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.314 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:39.314 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:39.314 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.314 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.314 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:39.314 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:39.314 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:39.315 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:39.315 INFO 1433467 --- [ForkJoinPool-1386575-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.315 INFO 1433467 --- [ForkJoinPool-1386575-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.315 INFO 1433467 --- [ForkJoinPool-1386575-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.315 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:39.317 INFO 1433467 --- [ForkJoinPool-1386575-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:39.317 INFO 1433467 --- [ForkJoinPool-1386575-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:55:39.318 INFO 1433467 --- [pool-ocp-schedules-18,22990421198c4e6f,3106c90f3332] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:55:39.319 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.320 INFO 1433467 --- [ForkJoinPool-1386575-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.320 INFO 1433467 --- [ForkJoinPool-1386575-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.320 INFO 1433467 --- [ForkJoinPool-1386575-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.320 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.320 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.321 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:39.322 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:39.323 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:39.324 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:39.324 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:39.324 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:39.325 INFO 1433467 --- [ForkJoinPool-1386575-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:39.325 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:39.326 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:39.326 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:39.329 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:39.329 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:39.330 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:39.330 INFO 1433467 --- [ForkJoinPool-1386575-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:55:39.330 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:39.330 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:39.331 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:39.334 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:55:39.335 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:55:39.336 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:55:39.336162679+08:00 2024-03-13 15:55:39.337 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:55:39.337 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:55:39.336+08:00 2024-03-13 15:55:39.338 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:55:39.337922586+08:00 2024-03-13 15:55:39.338 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:55:39.338 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:55:39.339 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:55:39.338+08:00 2024-03-13 15:55:39.339 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:55:39.339206709+08:00 2024-03-13 15:55:39.340 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:55:39.339+08:00 2024-03-13 15:55:39.340 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:55:39.336+08:00 and status AVAILABLE 2024-03-13 15:55:39.340 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:55:39.340238783+08:00 2024-03-13 15:55:39.341 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:55:39.340+08:00 2024-03-13 15:55:39.341 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:55:39.341 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:55:39.341029141+08:00 2024-03-13 15:55:39.342 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.342 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:55:39.341+08:00 2024-03-13 15:55:39.342 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:55:39.338+08:00 and status AVAILABLE 2024-03-13 15:55:39.343 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:55:39.343175301+08:00 2024-03-13 15:55:39.343 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:55:39.339+08:00 and status AVAILABLE 2024-03-13 15:55:39.344 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:55:39.341+08:00 and status AVAILABLE 2024-03-13 15:55:39.344 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:55:39.343+08:00 2024-03-13 15:55:39.344 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:55:39.340+08:00 and status AVAILABLE 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for agent11,22990421198c4e6f,5dc88d39bb4c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.345 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:55:39.346 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:55:39.346 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:55:39.343+08:00 and status AVAILABLE 2024-03-13 15:55:39.346 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:55:39.347 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:55:39.348 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:55:39.348 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:55:39.349 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:55:39.349 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:55:39.349 INFO 1433467 --- [pool-Check host task for agent25,22990421198c4e6f,12460fa9a190] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:55:39.350 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:55:39.350145324+08:00 2024-03-13 15:55:39.350 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.351 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:55:39.350+08:00 2024-03-13 15:55:39.351 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:55:39.351 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:55:39.351 INFO 1433467 --- [pool-Check host task for agent17,22990421198c4e6f,6bd00fb34c3c] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:55:39.351 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:55:39.352 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:55:39.350+08:00 and status AVAILABLE 2024-03-13 15:55:39.353 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:55:39.353 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:55:39.353 INFO 1433467 --- [pool-Check host task for agent6,22990421198c4e6f,586d02ce0ef8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:55:39.357 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:55:39.359 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.361 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.365 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:55:39.365 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:55:39.365 INFO 1433467 --- [pool-Check host task for agent24,22990421198c4e6f,93cc7a5aba09] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:55:39.367 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:55:39.367 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:55:39.367 INFO 1433467 --- [pool-Check host task for agent23,22990421198c4e6f,b8fb64159328] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:55:39.368 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:55:39.370 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:55:39.370 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:55:39.371 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:55:39.371 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:55:39.371 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:55:39.371 INFO 1433467 --- [pool-Check host task for agent8,22990421198c4e6f,95787d92c634] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:55:39.371 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:55:39.372 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:39.373 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:39.376 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:55:39.377 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:55:39.383 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.386 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:39.387 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:55:39.387 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:39.388 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.388 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:55:39.389 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.389 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:55:39.390 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:55:39.393 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.402 WARN 1433467 --- [pool-Check host task for clockdiff6,c2b88db545634026,147b8b80b94f] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.402 ERROR 1433467 --- [pool-Check host task for clockdiff6,c2b88db545634026,147b8b80b94f] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:55:39.403 WARN 1433467 --- [pool-Check host task for clockdiff6,c2b88db545634026,147b8b80b94f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.402 WARN 1433467 --- [pool-Check host task for clockdiff2,c2b88db545634026,b96c463cc429] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.403 ERROR 1433467 --- [pool-Check host task for clockdiff2,c2b88db545634026,b96c463cc429] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:55:39.403 WARN 1433467 --- [pool-Check host task for clockdiff2,c2b88db545634026,b96c463cc429] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.403 WARN 1433467 --- [pool-Check host task for clockdiff29,c2b88db545634026,2d127af644d8] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.403 ERROR 1433467 --- [pool-Check host task for clockdiff29,c2b88db545634026,2d127af644d8] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:55:39.403 WARN 1433467 --- [pool-Check host task for clockdiff29,c2b88db545634026,2d127af644d8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.420 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:55:39.421 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:55:39.424 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:55:39.428 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.430 WARN 1433467 --- [pool-Check host task for clockdiff24,c2b88db545634026,6e6a70bc9526] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.430 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:55:39.430 ERROR 1433467 --- [pool-Check host task for clockdiff24,c2b88db545634026,6e6a70bc9526] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:55:39.430 WARN 1433467 --- [pool-Check host task for clockdiff24,c2b88db545634026,6e6a70bc9526] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.430 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:55:39.433 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:55:39.434 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:55:39.438 WARN 1433467 --- [pool-Check host task for clockdiff8,c2b88db545634026,c485b623ec2c] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.438 ERROR 1433467 --- [pool-Check host task for clockdiff8,c2b88db545634026,c485b623ec2c] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:55:39.439 WARN 1433467 --- [pool-Check host task for clockdiff8,c2b88db545634026,c485b623ec2c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.439 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:55:39.449 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.449 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.450 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.451 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:55:39.454 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.470 WARN 1433467 --- [pool-Check host task for clockdiff1,c2b88db545634026,544fe2e944c0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.470 ERROR 1433467 --- [pool-Check host task for clockdiff1,c2b88db545634026,544fe2e944c0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:55:39.470 WARN 1433467 --- [pool-Check host task for clockdiff1,c2b88db545634026,544fe2e944c0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.484 WARN 1433467 --- [pool-Check host task for clockdiff3,c2b88db545634026,d1bb7f70dd99] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.484 ERROR 1433467 --- [pool-Check host task for clockdiff3,c2b88db545634026,d1bb7f70dd99] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:55:39.484 WARN 1433467 --- [pool-Check host task for clockdiff3,c2b88db545634026,d1bb7f70dd99] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:55:39.492 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:55:39.493 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:55:39.497 WARN 1433467 --- [pool-scanner-1609796195-1,7749f97dd8d6471a,42b2645bbd5e] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:39.499 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:55:39.499 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:55:39.499 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:55:39.506 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:55:39.506 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:55:39.507 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:55:39.508 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:55:39.509 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:55:39.510 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:55:39.512 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.513 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.514 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.514 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.514 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.514 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:55:39.515 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:55:39.518 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.518 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.518 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:55:39.524 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.525 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.527 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.533 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.533 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.533 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:55:39.533 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.533 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.534 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:39.535 WARN 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:55:39.535 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.535 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.535 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:55:39.536 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:55:39.537 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:39.540 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:55:39.541 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:55:39.543 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.543 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:39.553 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:55:39.554 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:55:39.554 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.554 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.557 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:55:39.558 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.561 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:55:39.561 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:55:39.561 INFO 1433467 --- [pool-Check host task for host54,c8a7656cdcc94f9b,f5f9b9a4cd2d] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:55:39.563 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.563 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.563 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.563 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.564 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:39.564 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:55:39.565 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:55:39.567 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:55:39.568 INFO 1433467 --- [pool-Check host task for host46,c8a7656cdcc94f9b,5339574ce8af] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:55:39.570 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:55:39.571 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:55:39.572 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:55:39.572 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.573 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:55:39.574 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:55:39.578 INFO 1433467 --- [pool-Check host task for host13,c8a7656cdcc94f9b,cf4e518f2903] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:55:39.579 INFO 1433467 --- [pool-Check host task for host44,c8a7656cdcc94f9b,0ecab956607b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:55:39.581 INFO 1433467 --- [pool-Check host task for host51,c8a7656cdcc94f9b,d7793bab94db] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:55:39.584 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:55:39.584 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:55:39.604 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:55:39.610 INFO 1433467 --- [pool-Check host task for host49,c8a7656cdcc94f9b,71d796e80840] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:55:39.618 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:55:39.623 INFO 1433467 --- [pool-Check host task for host45,c8a7656cdcc94f9b,0c410834610b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:55:39.665 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:55:39.673 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.673 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.673 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:55:39.673 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.677 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:55:39.682 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:55:39.682 INFO 1433467 --- [ForkJoinPool-1386574-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710315940072000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:55:39.723 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.724 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.724 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.724 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.724 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:55:39.749 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.749 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.750 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.750 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.751 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:55:39.755 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.755 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.755 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:55:39.760 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:39.772 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.772 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:39.789 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.789 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:39.804 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.804 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:39.817 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.817 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:55:39.825 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.825 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:55:39.833 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:55:39.843 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.843 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.843 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.843 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.844 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:55:39.848 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:55:39.852 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:55:39.855 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:55:39.857 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:55:39.877 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:55:39.899 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805159,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:39.911 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:55:39.925 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:55:39.926 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.926 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:55:39.926 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:55:39.927 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:55:39.938 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:55:39.938 INFO 1433467 --- [ForkJoinPool-1386574-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:55:40.068 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,08160f5ea5874bc7,21a2e21917d9] c.o.o.s.c.trace.RequestTracingAspect : API: [POST /services?Action=ObRootServiceInfo&User_ID=alibaba&UID=ocpmaster&ObRegion=obcluster&version=2&ObClusterId=1, client=10.10.10.148, traceId=08160f5ea5874bc7, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.updateRootServiceInfo(String,Long,String,Long,Integer,String), args=obcluster,1,2,{"ObRegion":"obcluster","ObCluster":"obcluster","ObRegionId":1,"ObClusterId":1,"Type":"PRIMARY","timestamp":1710316540066489,"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"ReadonlyRsList":[]},] 2024-03-13 15:55:40.083 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,08160f5ea5874bc7,21a2e21917d9] c.o.ocp.configurl.ConfigUrlService : updateRootServer, clusterName=obcluster, clusterId=1, affectRows=1 2024-03-13 15:55:40.084 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,08160f5ea5874bc7,21a2e21917d9] c.o.o.s.c.trace.RequestTracingAspect : API OK: [POST /services client=10.10.10.148, traceId=08160f5ea5874bc7, duration=16 ms] 2024-03-13 15:55:40.137 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:40.138 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:41.899 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805182,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:43.138 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:43.140 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:43.900 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805205,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:45.900 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805227,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:46.105 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805231,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:46.140 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:46.141 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:46.748 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:46.747565+08:00, lastProcessTime=2024-03-13T15:55:36.747607+08:00 2024-03-13 15:55:46.840 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:46.747565+08:00 2024-03-13 15:55:46.948 INFO 1433467 --- [pool-ocp-schedules-8,fef14601c0114840,4d5108179d2d] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:47.901 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805251,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:47.908 INFO 1433467 --- [pool-clean-time-series-data15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:55:47.924 INFO 1433467 --- [pool-print-time-series-interval15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316547: {1710316440=1394, 1710316455=1394, 1710316470=1394, 1710316485=1394, 1710316500=1394, 1710316515=1394, 1710316530=1394, 1710316545=60}, tsDataMapSize: 1122624, readyDataCacheSize: 8288 2024-03-13 15:55:47.949 INFO 1433467 --- [pool-clean-time-series-data60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:55:47.949 INFO 1433467 --- [pool-print-time-series-interval60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316547: {1710316380=34, 1710316440=32, 1710316500=6}, tsDataMapSize: 17768, readyDataCacheSize: 18488 2024-03-13 15:55:49.141 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:49.143 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:49.275 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805268,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:49.349 WARN 1433467 --- [pool-scanner-1609796811-1,167629ef5db04233,c447da5acebf] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:49.417 INFO 1433467 --- [alarm-detect-4,c5ee5d35b14b4195,af0c70a1884b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:55:49.419 INFO 1433467 --- [alarm-detect-4,c5ee5d35b14b4195,af0c70a1884b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316515 ~ 1710316539 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316530=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316515 ~ 1710316539 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316530=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316515 ~ 1710316539 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316530=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316515 ~ 1710316539 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316530=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316515 ~ 1710316539 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316530=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:55:49.463 INFO 1433467 --- [alarm-detect-26,cc9c12ecfabd4cea,ba8337f7864e] c.o.o.a.core.detect.AlarmRuleDetector : detectByMetric done, alarmType=host_ntp_service_not_exist, needSendingCount=1, detectAlarmsCount=1, matchedTargets=[alarm_template_id=3:host=10.10.10.148-2, alarm_template_id=3:host=10.10.10.146-2, alarm_template_id=3:host=10.10.10.145-2, alarm_template_id=3:host=10.10.10.144-2, alarm_template_id=3:host=10.10.10.143-2] 2024-03-13 15:55:49.901 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805275,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:51.902 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805298,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:52.143 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:52.144 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:53.176 INFO 1433467 --- [pool-ocp-schedules-8,761291c56aa14358,243a7aaa6071] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:55:53.184 INFO 1433467 --- [pool-ocp-schedules-8,761291c56aa14358,243a7aaa6071] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:53.184 INFO 1433467 --- [pool-ocp-schedules-8,761291c56aa14358,243a7aaa6071] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:55:53.902 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805321,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:55.144 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:55.145 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:55.713 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2024-03-13 15:55:55.713 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=609459010050985984 2024-03-13 15:55:55.717 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2024-03-13 15:55:55.717 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624101396131614721 2024-03-13 15:55:55.721 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2024-03-13 15:55:55.721 INFO 1433467 --- [pool-ocp-schedules-6,65248a2be9ab4ee6,c55d7ea3a408] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624425259050733570 2024-03-13 15:55:55.902 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805344,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:56.105 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805348,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:55:56.274 INFO 1433467 --- [pool-Check obproxy12,7ef44117d4f14915,330dae270639] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:55:56.276 INFO 1433467 --- [pool-Check obproxy13,7ef44117d4f14915,da50881edfbe] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:55:56.276 INFO 1433467 --- [pool-Check obproxy5,7ef44117d4f14915,62c8313d50c4] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:55:56.276 INFO 1433467 --- [pool-Check obproxy8,7ef44117d4f14915,7dd6bdf06d70] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:55:56.747 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:55:56.746743+08:00, lastProcessTime=2024-03-13T15:55:46.747565+08:00 2024-03-13 15:55:56.869 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:55:56.746743+08:00 2024-03-13 15:55:56.955 INFO 1433467 --- [pool-ocp-schedules-6,696dc1cdbbce4d5b,a6005d48e5ab] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:55:57.903 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805368,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:55:58.145 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:55:58.146 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:55:59.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805383,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:55:59.282 INFO 1433467 --- [alarm-detect-2,7c76b2805c9a477d,d5ac744cb4c4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:55:59.283 INFO 1433467 --- [alarm-detect-16,d96dbbb705b14f43,da0cc72a338f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:55:59.284 INFO 1433467 --- [alarm-detect-20,473880d2e31a47ad,541c2733932d] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:55:59.287 INFO 1433467 --- [alarm-detect-2,7c76b2805c9a477d,d5ac744cb4c4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:55:59.288 INFO 1433467 --- [alarm-detect-16,d96dbbb705b14f43,da0cc72a338f] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:55:59.289 INFO 1433467 --- [alarm-detect-20,473880d2e31a47ad,541c2733932d] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:55:59.333 WARN 1433467 --- [pool-scanner-1609797397-1,09650d6c7e6940b1,1cff461a346e] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:55:59.430 INFO 1433467 --- [alarm-detect-17,70107dc2a1b145d0,dec3aa7e245e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316530 ~ 1710316549 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316545=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:55:59.451 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805383,,] c.o.o.a.c.t.DistributedAlarmDetectTask : Refresh detector factory. 2024-03-13 15:55:59.903 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805391,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:00.566 INFO 1433467 --- [pool-prometheus-minute-schedule-1,7daf0c38589b4c3c,6432c89e7dd1] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1710316620, interval=60 2024-03-13 15:56:00.567 INFO 1433467 --- [pool-prometheus-cache-monitor1,e81a42b7262d4922,0a4aca0ebce0] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":12075,"minuteWriteQueue":0,"minuteReadCacheSize":12328,"secondWriteCacheSize":3328,"seriesIdKeySize":67930,"secondWriteQueue":0,"minuteWriteCacheSize":12393} 2024-03-13 15:56:00.567 INFO 1433467 --- [pool-prometheus-cache-monitor1,e81a42b7262d4922,0a4aca0ebce0] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{go_memory_classes_heap_stacks_bytes=15, ob_sysstat_40012=19, ob_server_resource_memory_assigned_bytes=13, ob_sysstat_40011=19, ob_sysstat_40010=19, ob_unit_num=13, ob_database_disk_required_size=35, go_memstats_mspan_sys_bytes=15, go_memstats_stack_sys_bytes=15, ob_tenant_request_queue_total_size=18, jvm_gc_memory_allocated_bytes_total=3, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=18, ob_sysstat_40019=19, observer_fd_count=13, ob_sysstat_40018=19, processor_process_metrics_total=21, jvm_gc_max_data_size_bytes=3, go_memstats_frees_total=15, sql_audit_input_collect_data_jumped_total=8, host_available=64, mysql_output_write_sql_failed_count=17, ob_database_disk_data_size=34, ob_system_event_detail_total_timeouts=763, ob_tenant_memstore_freeze_cnt=14, ob_latch_sleep_total=312, ob_tenant_unit_min_memory=25, ob_sysstat_60087=19, ob_memstore_total_bytes=20, metric_cardinality_count=12, process_resident_memory_bytes=15, jvm_threads_states_threads=8, ob_cache_size_bytes=34, ob_tenant_unit_max_memory=25, input_collect_metrics_total=26, ob_sysstat_60083=19, ob_sysstat_50039=19, ob_sysstat_50038=19, node_disk_io_time_weighted_seconds_total=33, process_open_fds=15, ob_sysstat_50037=19, process_memory_rss_bytes=33, ob_server_resource_cpu_assigned_percent=13, ob_sysstat_50030=19, go_memory_classes_profiling_buckets_bytes=15, go_memstats_heap_inuse_bytes=15, ob_sysstat_40118=19, ob_sysstat_40117=19, ob_sysstat_40116=19, node_memory_memfree_bytes=26, ob_sysstat_40005=19, process_exists=43, ob_sysstat_40004=19, ob_sysstat_40003=19, ob_sysstat_40002=19, ob_sysstat_40001=19, go_gc_duration_seconds=20, go_memory_classes_metadata_mspan_inuse_bytes=15, ob_server_resource_disk_bytes=13, ob_sysstat_40000=19, odp_current_session=26, process_thread_count=33, ob_sysstat_50040=19, ob_sysstat_40009=19, ob_sysstat_40008=19, ob_disk_free_bytes=13, node_filesystem_avail_bytes=72, ob_sysstat_40007=19, ob_sysstat_40006=19, ob_backup_storage_capacity_bytes=650, ob_unit_config_max_iops=21, ob_plan_cache_memory_bytes=18, node_network_transmit_errs_total=28, go_gc_duration_seconds_count=15, monagent_oceanbase_plan_explain_size=4, ob_sysstat_50011=19, ob_sysstat_50010=19, node_disk_read_bytes_total=33, ob_compaction_is_suspended=19, mysql_output_write_sql_milliseconds_summary_count=17, go_memstats_lookups_total=15, mysql_output_write_sql_count=17, process_cpu_percent=33, ob_sysstat_60057=19, host_agent_process_status=18, ob_server_resource_cpu=13, ob_tenant_server_data_index_row_count=20, go_memstats_alloc_bytes=15, go_sched_gomaxprocs_threads=15, node_cpu_seconds_total_native=38, ob_sysstat_140002=19, ob_sysstat_140005=19, ob_sysstat_140006=19, ob_sysstat_140003=19, ob_tenant_resource_min_memory=18, ob_memstore_freeze_times=20, ob_sysstat_50022=19, jvm_gc_pause_seconds_count=8, ob_sysstat_50021=19, jvm_gc_overhead_percent=3, go_memory_classes_heap_unused_bytes=15, ob_cluster_status_check=13, jvm_memory_committed_bytes=10, node_net_bandwidth_bps=29, monagent_oceanbase_db_pool_idle=4, ob_tenant_server_data_index_data_size=20, go_gc_heap_allocs_by_size_bytes_total_count=15, go_memstats_heap_idle_bytes=15, go_gc_cycles_forced_gc_cycles_total=15, go_gc_pauses_seconds_total_bucket=26, ob_server_resource_memory_bytes=13, ob_tenant_assigned_mem_assigned=18, ob_tenant_memstore_minor_freeze_times=14, ob_tenant_server_required_size=22, ob_sysstat_60031=19, process_fd_count=33, ob_tenant_unit_max_cpu=25, ob_latch_wait_seconds_total=312, go_memory_classes_metadata_mcache_free_bytes=15, node_memory_buffers_bytes=26, ob_sysstat_50009=19, go_memstats_heap_released_bytes=15, node_network_receive_bytes_total=28, ob_sysstat_50005=19, ob_sysstat_50008=19, ob_sysstat_50001=19, ob_sysstat_50004=19, exposer_request_latencies_count=6, ob_sysstat_50000=19, ob_session_active_num=18, mysql_output_context_timeout_discard_metrics_count=9, go_memory_classes_metadata_mcache_inuse_bytes=15, node_disk_read_time_seconds_total=33, ob_sysstat_60021=19, ob_sysstat_60020=19, ob_sysstat_60024=19, ob_compaction_frozen_version=19, ob_sysstat_60023=19, metric_count=15, ob_sysstat_60022=19, obproxysys_connectable=1556, ob_tenant_assigned_cpu_assigned=18, mysql_output_write_sql_retry_count=17, ob_tenant_unit_min_cpu=25, ob_unit_config_max_cpu=21, http_request_milliseconds_summary_count=75, monagent_oceanbase_audit_timeinfo_cache_size=4, http_request_milliseconds_summary=78, ob_latch_immediate_miss_total=312, ob_latch_spin_get_total=312, ob_log_level_count=17, monagent_pipeline_report_metrics_total=16, exposer_transferred_bytes_total=6, ob_tenant_context_memory_count=841, node_disk_reads_completed_total=33, ob_sysstat_60019=19, ob_sysstat_60003=19, ob_sysstat_60002=19, ob_sysstat_60001=19, ob_sysstat_60000=19, ob_latch_miss_total=312, go_sched_latencies_seconds_count=15, ocp_meta_db_health=5, ob_disk_total_bytes=13, go_memstats_heap_objects=15, go_memstats_mcache_inuse_bytes=15, ob_sysstat_60005=19, ob_sysstat_60004=19, go_memory_classes_other_bytes=15, ob_memstore_active_bytes=20, go_memstats_alloc_bytes_total=15, ob_tenant_server_partition_replica_count=21, process_count=34, ob_system_event_detail_total_waits=763, odp_used_connections=25, backup_storage_capacity_used_percent=8, go_gc_heap_frees_objects_total=15, go_memory_classes_metadata_mspan_free_bytes=15, ob_disk_log_used_bytes=13, ob_compaction_is_error=19, go_gc_limiter_last_enabled_gc_cycle=15, monagent_pipeline_execute_total=24, node_filesystem_files=86, process_start_time_seconds=15, go_memstats_heap_alloc_bytes=15, ob_tenant_resource_max_cpu=18, ntp_service_available=26, go_memstats_mspan_inuse_bytes=15, mysql_output_write_sql_milliseconds_summary=20, go_gc_heap_frees_by_size_bytes_total_count=15, monagent_pipeline_execute_seconds_total=24, ob_sysstat_80001=19, go_memstats_sys_bytes=15, go_memory_classes_heap_free_bytes=15, system_load_average_1m=3, monagent_oceanbase_audit_stat_cache_size=4, ob_tenant_memory_used_bytes=22, obproxy_server_max_connections=7, ob_sysstat_80002=19, go_memstats_gc_cpu_fraction=15, ob_sysstat_80003=19, monagent_oceanbase_audit_cache_size=4, ob_tenant_context_memory_used_bytes=841, ob_sysstat_80007=19, node_disk_writes_completed_total=33, ob_sysstat_80008=19, ob_sysstat_80009=19, go_sched_goroutines_goroutines=15, go_memory_classes_metadata_other_bytes=15, go_gc_cycles_total_gc_cycles_total=15, system_cpu_usage=3, process_cpu_usage=3, node_disk_written_bytes_total=33, monagent_oceanbase_ash_cache_size=4, go_goroutines=15, ob_sysstat_80023=19, exposer_request_latencies_sum=6, ob_unit_config_min_cpu=21, process_virtual_memory_max_bytes=15, go_gc_heap_frees_bytes_total=15, ob_tenant500_memory_hold_bytes=13, exposer_scrapes_total=6, go_gc_heap_allocs_by_size_bytes_total_bucket=27, ob_sysstat_80025=19, ob_sysstat_80026=19, oceanbase_connectivity=15, node_memory_sreclaimable_bytes=26, go_sched_latencies_seconds_sum=15, tcp_retrans=25, exporter_export_metrics_bytes_total=26, node_filesystem_size_bytes=72, http_server_requests_seconds_count=203, ob_compaction_frozen_time=19, monagent_input_sql_duration_seconds_sum=55, log_stream_degraded_count=18, ob_plan_cache_hit_total=18, node_filesystem_readonly=72, ob_sysstat_80040=19, ob_sysstat_80041=19, io_await=52, go_gc_heap_allocs_objects_total=15, go_memstats_mallocs_total=15, ob_unit_config_max_memory_bytes=21, go_gc_stack_starting_size_bytes=15, io_util=52, ob_sysstat_200002=19, ob_sysstat_200001=19, ob_tenant_disk_v4_data_size=18, go_sched_latencies_seconds_bucket=26, ob_tenant_assigned_cpu_total=18, ob_tenant_server_available_disk_free=19, node_disk_write_time_seconds_total=33, cpu_count=25, output_write_metrics_total=9, ob_cluster_sync=15, go_gc_heap_objects_objects=15, odp_transaction_total=16, ob_system_event_detail_time_waited_us=763, process_max_fds=15, http_status_count=20, ob_sysstat_80057=19, ocp_alarm_detect_duration_ms_count=115, go_gc_heap_allocs_by_size_bytes_total_sum=15, go_memstats_mcache_sys_bytes=15, ob_tenant_disk_v4_log_size=18, go_cgo_go_to_c_calls_calls_total=15, ocp_alarm_detect_duration_ms_sum=115, jvm_threads_live_threads=3, node_memory_cached_bytes=26, ob_table_num=16, ob_sysstat_10000=19, ob_sysstat_10006=19, ob_sysstat_10005=19, ob_sysstat_10002=19, ob_sysstat_10001=19, go_memstats_last_gc_time_seconds=15, ob_sysstat_10004=19, ob_sysstat_10003=19, monitor_exporter_avaliable=62, log_tailer_count=15, ob_server_num=25, ob_session_all_num=18, ob_system_event_time_waited=24, ob_tenant_resource_min_cpu=18, monagent_plugin_execute_seconds_total=39, go_gc_cycles_automatic_gc_cycles_total=15, go_gc_heap_allocs_bytes_total=15, go_memstats_gc_sys_bytes=15, ob_latch_get_total=312, go_info=16, log_tailer_processed_file_count=21, node_load15=24, ob_disk_log_assigned_bytes=13, go_memory_classes_heap_released_bytes=15, jvm_memory_max_bytes=10, ob_unit_config_min_iops=21, ob_sysstat_30080=19, ob_sysstat_30082=19, ob_sysstat_30081=19, ob_sysstat_30084=19, ob_sysstat_30083=19, ob_sysstat_30086=19, ob_sysstat_30085=19, ob_tenant_log_disk_total_bytes=18, ob_memstore_freeze_trigger_bytes=20, ob_waitevent_wait_seconds_total=18, ob_server_stopped_duration_seconds=15, go_memstats_next_gc_bytes=15, node_filesystem_files_free=86, ob_memtable_max_snapshot_duration_seconds=18, monagent_oceanbase_audit_unique_key_cache_size=4, process_virtual_memory_bytes=15, ob_sysstat_20002=19, net_connectivity=15, ob_sysstat_20001=19, monagent_input_sql_duration_seconds_bucket=65, ob_latch_immediate_get_total=312, node_cpu_seconds_total=32, ob_sysstat_20000=19, ob_index_error_num=13, ob_tenant_server_available_disk_total=19, ob_tenant_memory_hold_bytes=22, ob_sysstat_30066=19, go_memstats_buck_hash_sys_bytes=15, mysql_output_write_sql_milliseconds_summary_sum=17, ob_tenant_disk_total_size=18, odp_sql_request_total=53, mysql_output_write_task_size=17, exporter_export_metrics_total=26, ob_disk_log_total_bytes=13, system_cpu_count=3, ob_sysstat_130004=19, process_uptime_seconds=33, node_ntp_offset_seconds=25, go_memstats_heap_sys_bytes=15, ob_sysstat_140012=19, ob_sysstat_140013=19, jvm_gc_pause_seconds_max=8, ob_tenant_request_queue_large_size=18, ob_sysstat_130000=19, go_memstats_other_sys_bytes=15, ob_sysstat_130001=19, jvm_memory_used_bytes=10, ob_sysstat_130002=19, go_threads=15, http_request_milliseconds_summary_sum=75, http_server_requests_seconds_sum=203, ob_tenant_assigned_mem_total=18, node_memory_memtotal_bytes=26, node_network_transmit_drop_total=28, go_gc_pauses_seconds_total_count=15, exposer_request_latencies=9, ob_waitevent_wait_total=18, go_gc_heap_goal_bytes=15, ob_tenant_log_disk_used_bytes=18, jvm_gc_memory_promoted_bytes_total=3, odp_sql_cost_total=51, ob_sysstat_30011=19, ob_sysstat_30010=19, ob_sysstat_30013=19, ob_sysstat_30012=19, ob_tenant500_memory_used_bytes=13, node_network_transmit_bytes_total=28, node_network_receive_drop_total=28, go_gc_heap_tiny_allocs_objects_total=15, go_gc_duration_seconds_sum=15, ob_server_resource_memory_assigned_percent=13, jvm_gc_pause_seconds_sum=8, go_memory_classes_heap_objects_bytes=15, node_network_receive_errs_total=28, observer_ulimit_max_fd_count=13, ob_plan_cache_access_total=18, ob_tenant_server_row_count=22, monagent_oceanbase_plan_cache_size=4, ob_tenant_server_data_size=22, monagent_oceanbase_audit_sample_cache_size=4, go_gc_heap_frees_by_size_bytes_total_sum=15, ob_sysstat_40030=19, node_load1=24, process_cpu_seconds_total=15, ob_sysstat_120001=19, go_memory_classes_total_bytes=15, ob_sysstat_120002=19, ob_sysstat_120000=19, ob_tenant_resource_max_memory=18, ob_sysstat_120009=19, go_memstats_stack_inuse_bytes=15, ob_sysstat_120008=19, node_load5=24, ob_sysstat_120005=19, go_memory_classes_os_stacks_bytes=15, ob_sysstat_120006=19, jvm_threads_peak_threads=3, monagent_input_sql_duration_seconds_count=55, ob_tenant_context_memory_hold_bytes=841, obproxy_memory_limit_bytes=7, ob_compaction_last_version=19, ob_sysstat_30000=19, ob_server_resource_cpu_assigned=13, ob_sysstat_30002=19, ob_sysstat_30001=19, monagent_oceanbase_plan_timeinfo_cache_size=4, monagent_plugin_execute_total=39, ob_sysstat_30006=19, ob_sysstat_30005=19, ob_sysstat_30008=19, ob_sysstat_30007=19, ob_sysstat_30009=19, go_gc_heap_frees_by_size_bytes_total_bucket=27, obproxy_client_max_connections=7, ob_compaction_current_timestamp=19, ob_clog_max_sync_delay_seconds=20, go_gc_pauses_seconds_total_sum=15, ob_compaction_start_time=19} 2024-03-13 15:56:00.573 INFO 1433467 --- [pool-prometheus-minute-schedule-1,7daf0c38589b4c3c,6432c89e7dd1] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=18, collectTaskCount=18 2024-03-13 15:56:00.573 INFO 1433467 --- [pool-prometheus-minute-schedule-1,7daf0c38589b4c3c,6432c89e7dd1] c.o.ocp.monitor.OcpMonitorManager : Schedule interval metric too slow, from=1710316560, count=18, elapsed=1708606244013 2024-03-13 15:56:00.593 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:00.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:00 CST 2024 2024-03-13 15:56:00.593 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=38, name=Detect alarms on schedule, templateName=BackupScheduleAlarmTemplateFactory, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:55:00.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:00.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Detect alarms, nodeCount=1, taskCount=1 2024-03-13 15:56:00.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:00.596 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Detect alarms, taskDefinitionId=38, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:00.602 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:00.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:00 CST 2024 2024-03-13 15:56:00.602 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999975, name=OasSqlCuring, templateName=OasSqlCuringSchedule, scheduleRule=0 */2 * * * *, lastRunTime=2024-03-13 15:54:00.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:00.624 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : Split list is empty, name=OasSqlCuringSchedule, splitKey=cluster_ids, context=Argument{parallelIdx=-1, stringMap={}, listMap={cluster_ids=[]}} 2024-03-13 15:56:00.624 WARN 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule cron task failed. java.lang.RuntimeException: Illegal split list, the list is empty, splitKey=cluster_ids at com.oceanbase.ocp.core.metadb.repo.task.model.TaskInstanceEntity.generateSubtasks(TaskInstanceEntity.java:150) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.buildTaskInstance(ReadyTaskWorker.java:270) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.submitTask(ReadyTaskWorker.java:210) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.doScheduleTask(ReadyTaskWorker.java:197) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.checkAndScheduleCronTask(ReadyTaskWorker.java:173) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.lambda$loopScheduleTask$1(ReadyTaskWorker.java:133) at org.springframework.transaction.support.TransactionOperations.lambda$executeWithoutResult$0(TransactionOperations.java:68) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) at org.springframework.transaction.support.TransactionOperations.executeWithoutResult(TransactionOperations.java:67) at com.oceanbase.ocp.core.task.engine.coordinator.worker.AbstractTaskWorker.transactionWithoutResult(AbstractTaskWorker.java:62) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.loopScheduleTask(ReadyTaskWorker.java:132) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.work(ReadyTaskWorker.java:101) at com.oceanbase.ocp.core.task.engine.coordinator.TaskCoordinator.lockAndExecute(TaskCoordinator.java:94) at com.oceanbase.ocp.core.task.engine.coordinator.TaskCoordinator.lambda$startup$0(TaskCoordinator.java:65) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:00.629 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:01.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:00 CST 2024 2024-03-13 15:56:00.629 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999981, name=Collect all tenants major compactions, templateName=CollectTenantCompactionSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:55:01.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:00.638 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all tenants major compactions, nodeCount=1, taskCount=1 2024-03-13 15:56:00.638 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:00.638 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all tenants major compactions, taskDefinitionId=999981, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:00.644 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:01.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:00 CST 2024 2024-03-13 15:56:00.644 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999982, name=Collect SlowSql to SqlAggregateData for alarm, templateName=CollectSlowSqlSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:55:01.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:00.926 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect SlowSql to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:56:00.926 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:00.926 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect SlowSql to SqlAggregateData for alarm, taskDefinitionId=999982, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:00.932 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:01.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:00 CST 2024 2024-03-13 15:56:00.932 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999983, name=Collect trans stat to SqlAggregateData for alarm, templateName=CollectTransSatatSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:55:01.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:01.146 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:01.147 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:01.192 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect trans stat to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:56:01.192 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:01.192 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect trans stat to SqlAggregateData for alarm, taskDefinitionId=999983, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:01.197 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:01.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:01 CST 2024 2024-03-13 15:56:01.197 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999984, name=Get agent alarms, templateName=GetAgentAlarmsSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:55:01.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:01.200 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Get agent alarms, nodeCount=1, taskCount=1 2024-03-13 15:56:01.200 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:01.200 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Get agent alarms, taskDefinitionId=999984, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:01.207 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:54:01.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:01 CST 2024 2024-03-13 15:56:01.207 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999992, name=Sync root service list, templateName=SyncRsListSchedule, scheduleRule=0 */2 * * * ?, lastRunTime=2024-03-13 15:54:01.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:01.210 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Sync root service list, nodeCount=1, taskCount=2 2024-03-13 15:56:01.210 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:01.210 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:01.210 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Sync root service list, taskDefinitionId=999992, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:01.217 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:31.0, nextRun=2024-03-13 15:56:00.0, currentDate=Wed Mar 13 15:56:01 CST 2024 2024-03-13 15:56:01.217 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:55:31.0, nextRunTime=2024-03-13 15:56:00.0} 2024-03-13 15:56:01.218 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:56:01.219 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:01.219 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:01.496 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016747, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.512 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016748, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.521 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016747, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014397, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.493+08:00, sub_task_instance_name=Detect alarms of backup, sub_task_instance_id=10016747}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.524 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016749, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.531 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,d2294683914b] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016748, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014398, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.509+08:00, sub_task_instance_name=Collect all tenants major compactions, sub_task_instance_id=10016748}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.531 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,d2294683914b] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2024-03-13 15:56:01.543 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016750, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.544 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,6d25376f25c1] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016749, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014399, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.522+08:00, sub_task_instance_name=Collect SlowSql to SqlAggregateData for alarm, sub_task_instance_id=10016749}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.546 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.549 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:56:01.550 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:56:01.550 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:56:01.550 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:56:01.551 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.554 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.559 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.560 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.560 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:56:01.560 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.563 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.561 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016751, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.562 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.564 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,19212c62189b] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016750, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014400, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.539+08:00, sub_task_instance_name=Collect trans stat to SqlAggregateData for alarm, sub_task_instance_id=10016750}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.564 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:56:01.564 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.564 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:56:01.565 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.566 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.566 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.567 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.567 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:56:01.568 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266400687471944, broadcast scn:1710266400687471944, saved frozen scn:1710266400687471944 2024-03-13 15:56:01.568 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402011840869, broadcast scn:1710266402011840869, saved frozen scn:1710266402011840869 2024-03-13 15:56:01.568 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,0f55e3a68e5b] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:56:01.568 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,f5d0875c35f3] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:56:01.569 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402269282986, broadcast scn:1710266402269282986, saved frozen scn:1710266402269282986 2024-03-13 15:56:01.569 INFO 1433467 --- [pool-Collect all tenants major compactions9,f47a761474a84dce,71d815463f16] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:56:01.570 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266403004770377, broadcast scn:1710266403004770377, saved frozen scn:1710266403004770377 2024-03-13 15:56:01.570 INFO 1433467 --- [pool-Collect all tenants major compactions10,f47a761474a84dce,d59a5b54c370] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:56:01.570 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,d2294683914b] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2024-03-13 15:56:01.574 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,d2294683914b] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016748, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.577 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016752, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.579 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,e774edb5c5e2] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016751, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014401, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.557+08:00, sub_task_instance_name=Get all agent alarms, sub_task_instance_id=10016751}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.585 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,19212c62189b] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster1, trans stat result is empty. 2024-03-13 15:56:01.585 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,19212c62189b] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster, trans stat result is empty. 2024-03-13 15:56:01.587 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016753, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.588 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,6d25376f25c1] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for obcluster1, save 1 to SqlAggregateData 2024-03-13 15:56:01.589 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,19212c62189b] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016750, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.590 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,6d25376f25c1] c.o.o.p.s.i.alarm.SlowSqlDataSource : query sqlCount of obCluster obcluster, sql count is 0. 2024-03-13 15:56:01.589 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,6644df4cf228] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016752, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014402, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.575+08:00, sub_task_instance_name=Refresh root service list, sub_task_instance_id=10016752}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:56:01.590 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,6d25376f25c1] c.o.o.p.s.i.alarm.SlowSqlDataSource : query SlowSqlData of obCluster obcluster, result is empty. 2024-03-13 15:56:01.593 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,6d25376f25c1] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016749, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.596 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016754, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2024-03-13 15:56:01.599 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,6644df4cf228] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016752, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=7 2024-03-13 15:56:01.599 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,11075ac7b7fd] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016753, context=Context{parallelIdx=1, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014402, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.585+08:00, sub_task_instance_name=Refresh root service list, sub_task_instance_id=10016753}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:56:01.599 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=6 2024-03-13 15:56:01.601 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,e774edb5c5e2] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016751, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.602 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:01.602 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:01.602 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:01.603 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:01.603 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:01.603 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:01.603 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:01.604 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:01.604 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:01.605 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:01.605 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:01.607 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:01.607 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,11075ac7b7fd] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016753, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.607 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:01.608 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:01.614 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.615 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,8d2c3ae7cfab] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016754, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014403, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:01.594+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016754}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:01.616 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.618 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.620 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.622 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.623 INFO 1433467 --- [pool-Check obproxy15,f47a761474a84dce,d16b37d2323f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:56:01.623 INFO 1433467 --- [pool-Check obproxy10,f47a761474a84dce,57f7c8d91b14] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:56:01.624 INFO 1433467 --- [pool-Check obproxy9,f47a761474a84dce,681ad4e1d7d1] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:56:01.625 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.626 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62889/api/v1/log/alarms, params:null 2024-03-13 15:56:01.626 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,693a7d504e57] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:56:01.639 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.640 INFO 1433467 --- [pool-Check obproxy10,f47a761474a84dce,57f7c8d91b14] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:56:01.642 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.643 INFO 1433467 --- [pool-Check obproxy9,f47a761474a84dce,681ad4e1d7d1] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:56:01.644 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.646 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.648 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:56:01.650 INFO 1433467 --- [pool-Check obproxy15,f47a761474a84dce,d16b37d2323f] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:56:01.660 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,693a7d504e57] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:56:01.668 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,8d2c3ae7cfab] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016754, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:01.670 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:56:01.687 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.690 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.692 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.695 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:56:01.738 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.741 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.744 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.745 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.748 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:56:01.769 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:56:01.793 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.796 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.810 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.813 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.816 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.819 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.822 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:56:01.825 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:56:01.829 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:01.832 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.834 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [3600] 2024-03-13 15:56:01.877 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=3600, delayTasks=[] 2024-03-13 15:56:01.889 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.892 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.904 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805414,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:01.913 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:01.916 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.920 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.923 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.926 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:56:01.930 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:56:01.934 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:01.937 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:01.942 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [300] 2024-03-13 15:56:01.974 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=300, delayTasks=[] 2024-03-13 15:56:02.001 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:02.008 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:02.011 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:02.014 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:02.019 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [90] 2024-03-13 15:56:02.059 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=90, timeoutTasks=[] 2024-03-13 15:56:02.077 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:02.081 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:02.085 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:02.088 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:02.092 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [60] 2024-03-13 15:56:02.123 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=60, timeoutTasks=[] 2024-03-13 15:56:02.141 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:02.144 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:02.147 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:02.149 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:02.153 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:56:02.191 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:56:02.201 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:02.204 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:02.207 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:02.209 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:02.213 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:56:02.246 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:56:02.250 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster1 does not enable secondary backup 2024-03-13 15:56:02.253 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster does not enable secondary backup 2024-03-13 15:56:02.259 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackup/], metricValues size:[0]. 2024-03-13 15:56:02.263 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackupocp/], metricValues size:[0]. 2024-03-13 15:56:02.277 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,6754ea5666d3] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016747, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:02.360 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,2b64fd7cebab4e7c,e8ebfe11c707] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.148, traceId=2b64fd7cebab4e7c, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:56:02.365 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,2b64fd7cebab4e7c,e8ebfe11c707] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=2b64fd7cebab4e7c, duration=6 ms] 2024-03-13 15:56:02.565 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014397, status:SUCCESS 2024-03-13 15:56:02.566 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014397, status:SUCCESS 2024-03-13 15:56:02.577 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014398, status:SUCCESS 2024-03-13 15:56:02.578 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014398, status:SUCCESS 2024-03-13 15:56:02.586 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014399, status:SUCCESS 2024-03-13 15:56:02.586 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014399, status:SUCCESS 2024-03-13 15:56:02.594 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014400, status:SUCCESS 2024-03-13 15:56:02.595 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014400, status:SUCCESS 2024-03-13 15:56:02.603 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014401, status:SUCCESS 2024-03-13 15:56:02.603 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014401, status:SUCCESS 2024-03-13 15:56:02.612 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014402, status:SUCCESS 2024-03-13 15:56:02.613 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014402, status:SUCCESS 2024-03-13 15:56:02.623 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014403, status:SUCCESS 2024-03-13 15:56:02.624 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014403, status:SUCCESS 2024-03-13 15:56:03.904 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805437,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:04.147 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:04.147 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:05.904 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805460,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:06.106 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805463,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:56:06.749 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:56:06.748226+08:00, lastProcessTime=2024-03-13T15:55:56.746743+08:00 2024-03-13 15:56:06.864 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:56:06.748226+08:00 2024-03-13 15:56:06.961 INFO 1433467 --- [pool-ocp-schedules-13,bd73e7d2551c45c2,0cfa9d681651] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:56:07.148 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:07.149 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:07.905 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805484,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:09.248 INFO 1433467 --- [pool-ocp-schedules-20,c0b855f694d54a2c,56b898f4fe9d] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2024-03-13 15:56:09.258 INFO 1433467 --- [pool-ocp-schedules-6,8f0f5d1d129c4e1f,9cf07786e889] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:56:09.259 INFO 1433467 --- [pool-ocp-schedules-12,f1093abb6be84a53,77218090e281] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:56:09.259 INFO 1433467 --- [pool-ocp-schedules-16,9515dc0f2bda42c9,d5dbf7880c98] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:56:09.260 INFO 1433467 --- [pool-ocp-schedules-20,c0b855f694d54a2c,56b898f4fe9d] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2024-03-13 15:56:09.261 INFO 1433467 --- [pool-ocp-schedules-6,8f0f5d1d129c4e1f,9cf07786e889] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:56:09.262 INFO 1433467 --- [pool-ocp-schedules-12,f1093abb6be84a53,77218090e281] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:56:09.263 INFO 1433467 --- [pool-ocp-schedules-16,9515dc0f2bda42c9,d5dbf7880c98] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:56:09.269 INFO 1433467 --- [pool-ocp-schedules-1,dc32853a41db4822,34c3f025b955] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:56:09.269 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:56:09.269 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:56:09.269 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:56:09.270 INFO 1433467 --- [pool-ocp-schedules-11,9b3701f2c3914bc0,5d32e79126d4] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:56:09.270 INFO 1433467 --- [pool-Check host task for clockdiff27,9b3701f2c3914bc0,ccbfc4caa86f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:56:09.270 INFO 1433467 --- [pool-Check host task for clockdiff4,9b3701f2c3914bc0,581a53b72626] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:56:09.270 INFO 1433467 --- [pool-Check host task for clockdiff7,9b3701f2c3914bc0,9a586c3aa8ac] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:56:09.271 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:56:09.270 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:56:09.271 INFO 1433467 --- [pool-Check host task for clockdiff30,9b3701f2c3914bc0,20c359cd66f0] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:56:09.271 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:56:09.271 INFO 1433467 --- [pool-Check host task for clockdiff18,9b3701f2c3914bc0,09d972ffedae] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:56:09.271 INFO 1433467 --- [pool-Check host task for clockdiff28,9b3701f2c3914bc0,47a4314bdd1d] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:56:09.272 INFO 1433467 --- [pool-Check host task for clockdiff21,9b3701f2c3914bc0,a4fb5edbdefe] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:56:09.272 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:56:09.276 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:09.277 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805500,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:56:09.278 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:09.280 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:09.281 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:09.282 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:09.282 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:09.282 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:09.283 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:09.283 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:09.283 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:09.283 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:09.284 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:09.285 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:09.285 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:09.285 INFO 1433467 --- [pool-ocp-schedules-5,88fc9c4da24f4bb5,4d8c7b66ddc1] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:56:09.286 INFO 1433467 --- [pool-ocp-schedules-17,7054d8b52feb4e81,5d0127db3f2b] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:56:09.286 INFO 1433467 --- [pool-ocp-schedules-10,a23690bdba70422c,db352dd6173d] c.o.o.s.task.scheduled.ClusterSchedules : Check host version schedule. 2024-03-13 15:56:09.286 INFO 1433467 --- [pool-ocp-schedules-10,a23690bdba70422c,db352dd6173d] o.o.c.t.e.u.DistributedExclusiveExecutor : Empty item list, taskName=Check host task for host version, skip execute 2024-03-13 15:56:09.326 WARN 1433467 --- [pool-Check host task for clockdiff21,9b3701f2c3914bc0,a4fb5edbdefe] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.327 ERROR 1433467 --- [pool-Check host task for clockdiff21,9b3701f2c3914bc0,a4fb5edbdefe] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:56:09.327 WARN 1433467 --- [pool-Check host task for clockdiff21,9b3701f2c3914bc0,a4fb5edbdefe] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.327 WARN 1433467 --- [pool-Check host task for clockdiff7,9b3701f2c3914bc0,9a586c3aa8ac] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.327 ERROR 1433467 --- [pool-Check host task for clockdiff7,9b3701f2c3914bc0,9a586c3aa8ac] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:56:09.327 WARN 1433467 --- [pool-Check host task for clockdiff7,9b3701f2c3914bc0,9a586c3aa8ac] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.328 INFO 1433467 --- [pool-ocp-schedules-8,c90e4d43b634438d,5217090c5e7c] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:56:09.328 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:56:09.328 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:56:09.328 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:56:09.329 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:56:09.330 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:56:09.330 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:56:09.330 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:56:09.334 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:56:09.335 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:09.336 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:09.336 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:56:09.336488937+08:00 2024-03-13 15:56:09.337 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:56:09.337 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:09.337 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:09.337 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:56:09.336+08:00 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:09.338 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:09.339 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:09.339 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.339 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:09.339 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:56:09.338630900+08:00 2024-03-13 15:56:09.339 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:09.340 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:09.340 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.340 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.340 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:56:09.339876814+08:00 2024-03-13 15:56:09.340 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:56:09.340 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:56:09.339+08:00 2024-03-13 15:56:09.340 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:56:09.336+08:00 and status AVAILABLE 2024-03-13 15:56:09.341 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:09.341 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:56:09.340+08:00 2024-03-13 15:56:09.342 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:56:09.342124677+08:00 2024-03-13 15:56:09.342 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:56:09.342+08:00 2024-03-13 15:56:09.342 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:56:09.340+08:00 and status AVAILABLE 2024-03-13 15:56:09.343 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:56:09.339+08:00 and status AVAILABLE 2024-03-13 15:56:09.343 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:56:09.345 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:56:09.346 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:56:09.345246336+08:00 2024-03-13 15:56:09.346 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:56:09.346 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:56:09.342+08:00 and status AVAILABLE 2024-03-13 15:56:09.347 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:56:09.346+08:00 2024-03-13 15:56:09.348 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:56:09.347959669+08:00 2024-03-13 15:56:09.348 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:56:09.348 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:56:09.348 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:56:09.348+08:00 2024-03-13 15:56:09.350 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:56:09.346+08:00 and status AVAILABLE 2024-03-13 15:56:09.350 INFO 1433467 --- [ForkJoinPool-1386577-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.350 INFO 1433467 --- [ForkJoinPool-1386577-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.350 INFO 1433467 --- [ForkJoinPool-1386577-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.350 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:56:09.350620932+08:00 2024-03-13 15:56:09.351 INFO 1433467 --- [ForkJoinPool-1386577-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:09.351 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:56:09.351 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:56:09.351+08:00 2024-03-13 15:56:09.352 INFO 1433467 --- [ForkJoinPool-1386577-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.352 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:56:09.348+08:00 and status AVAILABLE 2024-03-13 15:56:09.353 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.353 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.353 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.353 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:09.354 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:56:09.354 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:56:09.351+08:00 and status AVAILABLE 2024-03-13 15:56:09.354 INFO 1433467 --- [ForkJoinPool-1386577-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.355 INFO 1433467 --- [ForkJoinPool-1386577-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.355 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:56:09.355 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:09.356 INFO 1433467 --- [ForkJoinPool-1386577-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:09.356 INFO 1433467 --- [ForkJoinPool-1386577-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:56:09.356 INFO 1433467 --- [ForkJoinPool-1386577-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:56:09.357 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:09.359 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:56:09.361 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.361 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:56:09.362 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:09.363 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:09.364 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:09.365 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:09.365 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:09.365 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.366 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:56:09.366 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:56:09.366 INFO 1433467 --- [pool-Check host task for agent9,c90e4d43b634438d,17d447c8cc41] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:56:09.366 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:09.366 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.367 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.367 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.367 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:09.368 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:09.368 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:09.368 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:56:09.369 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:56:09.369 INFO 1433467 --- [pool-Check host task for agent7,c90e4d43b634438d,d868e74e0dc2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:56:09.369 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:09.369 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:09.371 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:56:09.371 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:56:09.371 INFO 1433467 --- [pool-Check host task for agent3,c90e4d43b634438d,c202b76c9a69] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent1,c90e4d43b634438d,8092c975d757] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:56:09.372 INFO 1433467 --- [pool-Check host task for agent5,c90e4d43b634438d,fb99355cf3f3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:56:09.373 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.375 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:56:09.376 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:56:09.376 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:56:09.376 INFO 1433467 --- [pool-Check host task for agent19,c90e4d43b634438d,fc09d53934c9] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:56:09.377 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.378 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:56:09.379 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:56:09.379 INFO 1433467 --- [pool-Check host task for agent27,c90e4d43b634438d,fb3c78d036c3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:56:09.383 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.384 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.386 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.387 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.389 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.389 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:09.413 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:56:09.413 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:56:09.413 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:56:09.414 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:56:09.424 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:56:09.428 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:56:09.433 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:56:09.440 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:56:09.443 WARN 1433467 --- [pool-Check host task for clockdiff27,9b3701f2c3914bc0,ccbfc4caa86f] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.443 WARN 1433467 --- [pool-Check host task for clockdiff4,9b3701f2c3914bc0,581a53b72626] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.443 ERROR 1433467 --- [pool-Check host task for clockdiff27,9b3701f2c3914bc0,ccbfc4caa86f] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:56:09.443 ERROR 1433467 --- [pool-Check host task for clockdiff4,9b3701f2c3914bc0,581a53b72626] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:56:09.443 WARN 1433467 --- [pool-Check host task for clockdiff27,9b3701f2c3914bc0,ccbfc4caa86f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.443 WARN 1433467 --- [pool-Check host task for clockdiff4,9b3701f2c3914bc0,581a53b72626] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.445 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:56:09.446 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:56:09.447 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:56:09.448 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:56:09.455 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:56:09.456 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.457 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:56:09.458 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.462 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.467 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.467 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.468 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.468 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.469 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:56:09.469 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.470 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,a3f4636454e24191,6c66d809965e] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.143, traceId=a3f4636454e24191, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:56:09.471 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.473 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,a3f4636454e24191,6c66d809965e] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.143, traceId=a3f4636454e24191, duration=4 ms] 2024-03-13 15:56:09.476 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.487 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:56:09.487 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff28,9b3701f2c3914bc0,47a4314bdd1d] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.488 ERROR 1433467 --- [pool-Check host task for clockdiff28,9b3701f2c3914bc0,47a4314bdd1d] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff28,9b3701f2c3914bc0,47a4314bdd1d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff30,9b3701f2c3914bc0,20c359cd66f0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.488 ERROR 1433467 --- [pool-Check host task for clockdiff30,9b3701f2c3914bc0,20c359cd66f0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff30,9b3701f2c3914bc0,20c359cd66f0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff18,9b3701f2c3914bc0,09d972ffedae] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.488 ERROR 1433467 --- [pool-Check host task for clockdiff18,9b3701f2c3914bc0,09d972ffedae] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:56:09.488 WARN 1433467 --- [pool-Check host task for clockdiff18,9b3701f2c3914bc0,09d972ffedae] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:09.493 WARN 1433467 --- [pool-scanner-1609798002-1,fd4510bf4e7545c8,17e3475f8160] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:56:09.494 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:56:09.495 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:56:09.499 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:56:09.502 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:56:09.503 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:56:09.507 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:09.512 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:56:09.512 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.513 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.513 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:56:09.514 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:56:09.519 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:56:09.519 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.521 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:56:09.522 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.523 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.523 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.523 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.524 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.524 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:09.525 WARN 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:56:09.525 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.525 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.526 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:56:09.526 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:09.527 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:56:09.528 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:56:09.529 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.530 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.530 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:09.533 INFO 1433467 --- [pool-Check host task for host24,dc32853a41db4822,8a6902b9f66c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:56:09.534 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:56:09.534 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.536 INFO 1433467 --- [pool-Check host task for host56,dc32853a41db4822,1d62a786a44e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:56:09.539 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.541 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:56:09.547 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.547 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.548 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.548 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.548 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:09.548 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:56:09.550 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:56:09.553 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:09.553 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:09.554 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:09.554 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:56:09.558 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:56:09.563 INFO 1433467 --- [pool-Check host task for host50,dc32853a41db4822,20964c042b0a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:56:09.566 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:56:09.566 INFO 1433467 --- [pool-Check host task for host1,dc32853a41db4822,496c30e933d6] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:56:09.568 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:56:09.571 INFO 1433467 --- [pool-Check host task for host52,dc32853a41db4822,4f57fa219226] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:56:09.574 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:56:09.575 INFO 1433467 --- [pool-Check host task for host58,dc32853a41db4822,afc7c0bc4580] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:56:09.583 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:56:09.626 INFO 1433467 --- [pool-Check host task for host39,dc32853a41db4822,4328eeb0c128] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:56:09.634 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:56:09.648 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.648 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.648 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:09.648 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.649 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:56:09.656 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:56:09.656 INFO 1433467 --- [ForkJoinPool-1386578-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316540069000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:56:09.669 INFO 1433467 --- [alarm-detect-2,f35a7e4e7c494233,2e6b1e2a7c2c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000912, data is empty. 2024-03-13 15:56:09.672 INFO 1433467 --- [alarm-detect-2,f35a7e4e7c494233,2e6b1e2a7c2c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:56:09.682 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.683 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.683 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.683 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.683 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:56:09.711 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.711 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.711 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.711 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.712 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:09.715 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.715 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.715 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:56:09.718 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:09.724 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.724 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:09.731 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.731 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:09.737 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.737 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:09.744 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.744 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:09.750 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.750 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:09.758 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:09.776 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.777 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.777 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.777 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.778 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:09.782 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:56:09.787 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:56:09.792 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:09.794 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:09.811 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:56:09.844 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:56:09.863 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:09.864 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.864 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:09.864 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:09.865 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:56:09.877 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:56:09.877 INFO 1433467 --- [ForkJoinPool-1386578-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:56:09.905 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805509,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:10.149 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:10.149 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:10.401 INFO 1433467 --- [pool-async-queue-offer-interval60-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 60, queue size = 6, ruleSize = 6. 2024-03-13 15:56:10.403 INFO 1433467 --- [pool-async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 17, ruleSize = 17. 2024-03-13 15:56:10.858 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:10.858 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:10.867 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:10.867 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:10.872 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:10.872 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:10.877 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:10.877 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:11.087 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.087 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:56:11.087 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:56:11.087 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:56:11.088 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:56:11.089 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.090 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:56:11.091 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:56:11.115 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:56:11.116 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:56:11.117 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:56:11.117 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:56:11.135 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:56:11.136 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:56:11.137 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:56:11.137 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.154 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:56:11.155 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.162 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:56:11.169 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:56:11.170 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:56:11.170 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:56:11.170 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total 2024-03-13 15:56:11.170 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_route 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_server_process_request_time 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_hit 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_insert 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_update 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_response 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_sequence 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_partition 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_table 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_request 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_request 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_response 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_commit 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_delete 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_update 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_select 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_slow 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_others 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_sequence 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_large 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_prepare_send_request_time 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_success 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_select 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_delete 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_commit 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_others 2024-03-13 15:56:11.174 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_insert 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.176 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.178 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.180 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.181 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.181 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.181 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.181 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.181 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.183 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.185 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.186 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.186 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.186 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.186 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.186 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.188 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.190 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.192 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.193 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:56:11.195 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:56:11.906 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805532,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:12.279 INFO 1433467 --- [pool-metric-archive-second-5,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316500, nodeCount=1000, affectRows=1000 2024-03-13 15:56:12.283 INFO 1433467 --- [pool-metric-archive-second-5,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316500, nodeCount=1000, validCount=1000 2024-03-13 15:56:12.288 INFO 1433467 --- [pool-metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316500, nodeCount=333, affectRows=333 2024-03-13 15:56:12.289 INFO 1433467 --- [pool-metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316500, nodeCount=333, validCount=333 2024-03-13 15:56:12.298 INFO 1433467 --- [pool-metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316500, nodeCount=695, affectRows=695 2024-03-13 15:56:12.306 INFO 1433467 --- [pool-metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316500, nodeCount=695, validCount=695 2024-03-13 15:56:12.309 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316500, nodeCount=993, affectRows=993 2024-03-13 15:56:12.311 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316500, nodeCount=992, validCount=992 2024-03-13 15:56:13.150 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:13.151 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:13.186 INFO 1433467 --- [pool-ocp-schedules-11,a2a644a4427e43cc,b78e37ccddbe] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:56:13.194 INFO 1433467 --- [pool-ocp-schedules-11,a2a644a4427e43cc,b78e37ccddbe] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:56:13.194 INFO 1433467 --- [pool-ocp-schedules-11,a2a644a4427e43cc,b78e37ccddbe] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:56:13.906 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805556,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:14.629 INFO 1433467 --- [pool-metric-archive-async-8,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1710316500, nodeCount=2, affectRows=2 2024-03-13 15:56:14.636 INFO 1433467 --- [pool-metric-archive-async-8,,] c.o.o.m.service.OcpMetricArchiveService : Rollup async second to minute, timestamp=1710316500, nodeCount=2, validCount=2 2024-03-13 15:56:15.103 INFO 1433467 --- [pool-prometheus-second-schedule-1,ad0d3266a49947c8,0d53329e9cb3] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=23, collectTaskCount=23 2024-03-13 15:56:15.907 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805579,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:16.106 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805582,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:56:16.151 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:16.152 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:16.747 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:56:16.746917+08:00, lastProcessTime=2024-03-13T15:56:06.748226+08:00 2024-03-13 15:56:16.829 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:56:16.746917+08:00 2024-03-13 15:56:16.968 INFO 1433467 --- [pool-ocp-schedules-7,c4e5766acfba45cb,9ada81bc760d] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:56:17.907 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805603,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:19.152 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:19.153 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:19.276 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805618,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:56:19.286 INFO 1433467 --- [alarm-detect-7,df8c07d1b81547d8,2240b486dc48] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000912, data is empty. 2024-03-13 15:56:19.287 INFO 1433467 --- [alarm-detect-25,dda16d2c562044bd,850ae50d031e] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000912, data is empty. 2024-03-13 15:56:19.290 INFO 1433467 --- [alarm-detect-16,302903dbd7884129,229e6106e6d7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000912, data is empty. 2024-03-13 15:56:19.291 INFO 1433467 --- [alarm-detect-7,df8c07d1b81547d8,2240b486dc48] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:56:19.291 INFO 1433467 --- [alarm-detect-25,dda16d2c562044bd,850ae50d031e] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:56:19.293 INFO 1433467 --- [alarm-detect-16,302903dbd7884129,229e6106e6d7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:56:19.315 WARN 1433467 --- [pool-scanner-1609799659-1,8e626a88e6924ce3,4b94be5981b1] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:56:19.415 INFO 1433467 --- [alarm-detect-10,accabf20acd242b8,635b08f61642] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316545 ~ 1710316569 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316560=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:56:19.415 INFO 1433467 --- [alarm-detect-10,accabf20acd242b8,635b08f61642] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316545 ~ 1710316569 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316560=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:56:19.415 INFO 1433467 --- [alarm-detect-10,accabf20acd242b8,635b08f61642] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316545 ~ 1710316569 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316560=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:56:19.415 INFO 1433467 --- [alarm-detect-10,accabf20acd242b8,635b08f61642] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316545 ~ 1710316569 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316560=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:56:19.908 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805626,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:20.887 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:20.0, nextRun=2024-03-13 15:56:20.0, currentDate=Wed Mar 13 15:56:20 CST 2024 2024-03-13 15:56:20.888 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999996, name=Collect all cluster compaction info, templateName=CollectClusterCompactionSchedule, scheduleRule=20 * * * * *, lastRunTime=2024-03-13 15:55:20.0, nextRunTime=2024-03-13 15:56:20.0} 2024-03-13 15:56:20.890 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all cluster compaction info, nodeCount=1, taskCount=1 2024-03-13 15:56:20.891 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:20.891 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all cluster compaction info, taskDefinitionId=999996, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:21.783 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016755, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:21.804 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a4cd91395a7c] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016755, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014404, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:21.780+08:00, sub_task_instance_name=Collect compaction for all clusters, sub_task_instance_id=10016755}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:21.806 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a4cd91395a7c] c.o.o.s.t.b.c.AbstractAllClusterTask : Collect compaction for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:56:21.807 INFO 1433467 --- [ForkJoinPool-1386583-worker-1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:56:21.807 INFO 1433467 --- [ForkJoinPool-1386583-worker-2,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:56:21.815 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,a4cd91395a7c] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016755, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:21.908 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805649,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:22.031 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014404, status:SUCCESS 2024-03-13 15:56:22.032 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014404, status:SUCCESS 2024-03-13 15:56:22.153 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:22.154 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:22.331 INFO 1433467 --- [pool-ocp-schedules-11,32711f98b3c5444f,7d8d814f0e33] c.o.ocp.core.cache.helper.ObQueryCache : Init ob cache. 2024-03-13 15:56:22.344 INFO 1433467 --- [pool-ocp-schedules-11,32711f98b3c5444f,7d8d814f0e33] c.o.ocp.core.cache.helper.ObQueryCache : Finished init ob cache. 2024-03-13 15:56:22.815 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,570efb35811f4b51,928ce3e6c048] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.142, traceId=570efb35811f4b51, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:56:22.820 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,570efb35811f4b51,928ce3e6c048] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.142, traceId=570efb35811f4b51, duration=6 ms] 2024-03-13 15:56:23.665 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,c67902ce2c204210,2a9a7f1ba96b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.144, traceId=c67902ce2c204210, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:56:23.670 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,c67902ce2c204210,2a9a7f1ba96b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.144, traceId=c67902ce2c204210, duration=6 ms] 2024-03-13 15:56:23.909 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805672,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:25.155 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:25.155 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:25.909 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805695,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:26.107 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805698,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:56:26.341 INFO 1433467 --- [pool-Check obproxy7,12ab4a6ab3e54d10,4f070ed55d6b] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:56:26.342 INFO 1433467 --- [pool-Check obproxy2,12ab4a6ab3e54d10,b50bee226dbe] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:56:26.344 INFO 1433467 --- [pool-Check obproxy3,12ab4a6ab3e54d10,3dc9bdfbe902] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:56:26.345 INFO 1433467 --- [pool-Check obproxy6,12ab4a6ab3e54d10,58c32f765592] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:56:26.742 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:56:26.741760+08:00, lastProcessTime=2024-03-13T15:56:16.746917+08:00 2024-03-13 15:56:26.828 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:56:26.741760+08:00 2024-03-13 15:56:26.974 INFO 1433467 --- [pool-ocp-schedules-19,d7406e5f48044000,35a9729c8ca6] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:56:27.910 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805719,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:28.156 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:28.157 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:29.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805735,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:56:29.331 WARN 1433467 --- [pool-scanner-1609800315-1,a510478317234dc5,b91d2f7f1ae0] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:56:29.417 INFO 1433467 --- [alarm-detect-25,ff499c4dec1a4259,989d160f2c93] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:56:29.419 INFO 1433467 --- [alarm-detect-25,ff499c4dec1a4259,989d160f2c93] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:56:29.457 INFO 1433467 --- [alarm-detect-26,b2d5545ccc1643bd,c633c098b0e3] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316560 ~ 1710316579 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316575=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:56:29.457 INFO 1433467 --- [alarm-detect-26,b2d5545ccc1643bd,c633c098b0e3] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316560 ~ 1710316579 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316575=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:56:29.457 INFO 1433467 --- [alarm-detect-26,b2d5545ccc1643bd,c633c098b0e3] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316560 ~ 1710316579 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316575=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:56:29.910 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805743,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:30.734 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:55:31.0, nextRun=2024-03-13 15:56:30.0, currentDate=Wed Mar 13 15:56:30 CST 2024 2024-03-13 15:56:30.734 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999990, name=Sync tenant information, templateName=TenantSyncSchedule, scheduleRule=30 0/1 * * * ?, lastRunTime=2024-03-13 15:55:31.0, nextRunTime=2024-03-13 15:56:30.0} 2024-03-13 15:56:30.738 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Sync tenant information, nodeCount=1, taskCount=2 2024-03-13 15:56:30.738 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:30.738 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:30.738 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Sync tenant information, taskDefinitionId=999990, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:30.746 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:01.0, nextRun=2024-03-13 15:56:30.0, currentDate=Wed Mar 13 15:56:30 CST 2024 2024-03-13 15:56:30.746 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:56:01.0, nextRunTime=2024-03-13 15:56:30.0} 2024-03-13 15:56:30.747 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:56:30.747 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:56:30.747 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:56:30.869 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016756, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:30.879 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016757, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:30.881 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016756, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014405, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:30.867+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016756}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:56:30.890 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016758, operation:EXECUTE, state: RUNNING 2024-03-13 15:56:30.898 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016757, context=Context{parallelIdx=1, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014405, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:30.877+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016757}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:56:30.911 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,b271c2444a22] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016758, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014406, task_operation=execute, latest_execution_start_time=2024-03-13T15:56:30.888+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016758}, listMap={}}, executor=10.10.10.148 2024-03-13 15:56:30.922 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,5ac4319afbd5] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:56:30.923 INFO 1433467 --- [pool-Check obproxy13,f47a761474a84dce,ce7851ed2fff] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:56:30.923 INFO 1433467 --- [pool-Check obproxy11,f47a761474a84dce,4fbcc2fe6482] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:56:30.925 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.925 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:56:30.925 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,f57cb2525d30] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:56:30.926 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.926 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.926 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:56:30.932 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.932 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:56:30.932 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.932 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.932 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:56:30.935 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.935 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.936 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:30.937 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.937 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.937 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:30.942 INFO 1433467 --- [pool-Check obproxy13,f47a761474a84dce,ce7851ed2fff] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:56:30.943 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.943 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.944 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:30.946 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.946 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.946 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:30.947 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,5ac4319afbd5] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:56:30.952 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.952 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.952 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:30.954 INFO 1433467 --- [pool-Check obproxy11,f47a761474a84dce,4fbcc2fe6482] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:56:30.956 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,f57cb2525d30] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:56:30.959 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.959 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.960 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:30.960 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.960 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.960 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:30.965 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,b271c2444a22] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016758, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:30.969 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.969 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.969 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:30.981 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:30.981 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.982 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.982 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.985 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:56:30.991 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:30.992 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.992 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.992 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.992 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:56:30.996 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:30.997 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.997 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:30.997 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.997 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:56:30.998 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:30.999 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.999 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:30.999 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:30.999 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:56:31.006 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.006 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.006 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.006 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.007 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:56:31.010 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.010 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.010 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.010 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.010 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:56:31.015 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.016 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.016 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.016 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.017 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:56:31.017 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.017 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.017 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.018 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.018 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:56:31.023 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.023 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.024 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.024 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.024 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:56:31.024 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=2 2024-03-13 15:56:31.026 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:56:31.037 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.037 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.038 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.038 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.038 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:31.041 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.042 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.042 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.042 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.043 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:56:31.044 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=4, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:56:31.045 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=4, userWhitelist=% 2024-03-13 15:56:31.046 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:31.046 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.046 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:31.052 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=3, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:56:31.052 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.052 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=3, userWhitelist=% 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=2 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.053 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:56:31.056 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,d88ff64ad10c] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016757, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:31.069 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.069 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.069 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.069 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.070 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:56:31.072 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2024-03-13 15:56:31.075 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:56:31.093 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:31.093 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.094 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.094 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.095 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:56:31.104 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:56:31.107 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2024-03-13 15:56:31.107 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:31.107 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:31.108 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:56:31.113 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:56:31.115 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2024-03-13 15:56:31.115 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2024-03-13 15:56:31.115 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:56:31.120 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,8b306503d888] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016756, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:56:31.157 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:31.158 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:31.742 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014405, status:SUCCESS 2024-03-13 15:56:31.743 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014405, status:SUCCESS 2024-03-13 15:56:31.755 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014406, status:SUCCESS 2024-03-13 15:56:31.757 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014406, status:SUCCESS 2024-03-13 15:56:31.910 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805766,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:33.197 INFO 1433467 --- [pool-ocp-schedules-15,f916a5d7c64f4d79,3c3433f80e67] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:56:33.204 INFO 1433467 --- [pool-ocp-schedules-15,f916a5d7c64f4d79,3c3433f80e67] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:56:33.204 INFO 1433467 --- [pool-ocp-schedules-15,f916a5d7c64f4d79,3c3433f80e67] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:56:33.911 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805789,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:34.158 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:34.159 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:35.911 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805812,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:36.107 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805814,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:56:36.750 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:56:36.749831+08:00, lastProcessTime=2024-03-13T15:56:26.741760+08:00 2024-03-13 15:56:36.841 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:56:36.749831+08:00 2024-03-13 15:56:36.981 INFO 1433467 --- [pool-ocp-schedules-13,9459890a6c3d4039,8131fd33b2fb] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:56:37.159 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:37.160 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:37.912 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805835,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:38.607 INFO 1433467 --- [ForkJoinPool.commonPool-worker-7,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:38.608 INFO 1433467 --- [ForkJoinPool.commonPool-worker-7,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:38.608 INFO 1433467 --- [ForkJoinPool.commonPool-worker-7,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:38.608 INFO 1433467 --- [ForkJoinPool.commonPool-worker-7,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT SVR_IP, SVR_PORT, ZONE, CPU_CAPACITY, CPU_CAPACITY_MAX, CPU_ASSIGNED, CPU_ASSIGNED_MAX, MEM_CAPACITY, MEM_CAPACITY AS MEM_CAPACITY_MAX, MEM_ASSIGNED, MEM_ASSIGNED AS MEM_ASSIGNED_MAX, LOG_DISK_CAPACITY, LOG_DISK_ASSIGNED, LOG_DISK_IN_USE, DATA_DISK_CAPACITY, DATA_DISK_IN_USE FROM oceanbase.GV$OB_SERVERS 2024-03-13 15:56:38.617 INFO 1433467 --- [ForkJoinPool.commonPool-worker-0,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:38.617 INFO 1433467 --- [ForkJoinPool.commonPool-worker-0,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:38.617 INFO 1433467 --- [ForkJoinPool.commonPool-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:38.618 INFO 1433467 --- [ForkJoinPool.commonPool-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT SVR_IP, SVR_PORT, ZONE, CPU_CAPACITY, CPU_CAPACITY_MAX, CPU_ASSIGNED, CPU_ASSIGNED_MAX, MEM_CAPACITY, MEM_CAPACITY AS MEM_CAPACITY_MAX, MEM_ASSIGNED, MEM_ASSIGNED AS MEM_ASSIGNED_MAX, LOG_DISK_CAPACITY, LOG_DISK_ASSIGNED, LOG_DISK_IN_USE, DATA_DISK_CAPACITY, DATA_DISK_IN_USE FROM oceanbase.GV$OB_SERVERS 2024-03-13 15:56:38.767 INFO 1433467 --- [pool-ocp-schedules-19,47203c1815264302,f189e4463c35] c.o.o.c.cache.helper.ObProxyQueryCache : Init obproxy cache. 2024-03-13 15:56:38.779 INFO 1433467 --- [pool-ocp-schedules-19,47203c1815264302,f189e4463c35] c.o.o.c.cache.helper.ObProxyQueryCache : Finished init obproxy cache. 2024-03-13 15:56:39.255 INFO 1433467 --- [pool-ocp-schedules-6,2ada8c597e184c1c,abb4f73366a0] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:56:39.256 INFO 1433467 --- [pool-ocp-schedules-9,3720c6b7277341c2,e62818442140] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:56:39.257 INFO 1433467 --- [pool-ocp-schedules-5,636afd317a704877,b18c0af1614c] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:56:39.257 INFO 1433467 --- [pool-ocp-schedules-6,2ada8c597e184c1c,abb4f73366a0] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:56:39.259 INFO 1433467 --- [pool-ocp-schedules-9,3720c6b7277341c2,e62818442140] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:56:39.260 INFO 1433467 --- [pool-ocp-schedules-5,636afd317a704877,b18c0af1614c] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:56:39.268 INFO 1433467 --- [pool-ocp-schedules-8,12c93531f5a54402,d70a20191936] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:56:39.268 INFO 1433467 --- [pool-Check host task for clockdiff9,12c93531f5a54402,2adeb966527d] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff20,12c93531f5a54402,64016365eb68] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff15,12c93531f5a54402,6babde3707ec] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff12,12c93531f5a54402,7d2138e338a7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff16,12c93531f5a54402,7227dbeab2dd] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff23,12c93531f5a54402,6604970ce0df] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:56:39.269 INFO 1433467 --- [pool-Check host task for clockdiff17,12c93531f5a54402,d254bee2f074] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:56:39.270 INFO 1433467 --- [pool-ocp-schedules-13,15c7610b60d249a8,a1c6ba6fcf4e] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:56:39.270 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:56:39.270 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:56:39.271 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:56:39.271 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:56:39.271 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:56:39.271 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:56:39.271 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:56:39.276 INFO 1433467 --- [pool-ocp-schedules-15,068f4ceb52084afb,10125273c311] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:56:39.276 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:39.277 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:39.277 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:39.278 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:39.279 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:39.279 INFO 1433467 --- [pool-ocp-schedules-4,cae93a00a1794d21,2dd0f7d06ee0] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:56:39.279 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:39.279 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:39.280 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:39.280 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:39.281 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:39.282 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:39.283 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:39.283 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:39.283 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:39.283 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805851,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:56:39.288 INFO 1433467 --- [alarm-detect-15,9c0adecadd1f4d58,f6f125a209fa] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:56:39.291 INFO 1433467 --- [alarm-detect-20,929ca6d209f14a06,f67b2a3ba7c9] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:56:39.292 INFO 1433467 --- [alarm-detect-15,9c0adecadd1f4d58,f6f125a209fa] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-ocp-schedules-18,d505c6875a514ae0,90f3581de03d] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:56:39.294 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:56:39.295 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:56:39.295 INFO 1433467 --- [alarm-detect-16,5cdda404f60f45ca,d821a25dd492] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:56:39.300 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:39.301 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:39.303 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:39.326 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:39.327 WARN 1433467 --- [pool-Check host task for clockdiff20,12c93531f5a54402,64016365eb68] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.327 ERROR 1433467 --- [pool-Check host task for clockdiff20,12c93531f5a54402,64016365eb68] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:56:39.327 WARN 1433467 --- [pool-Check host task for clockdiff20,12c93531f5a54402,64016365eb68] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.328 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:39.329 INFO 1433467 --- [alarm-detect-16,5cdda404f60f45ca,d821a25dd492] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:56:39.329 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:39.330 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:39.332 INFO 1433467 --- [ForkJoinPool-1386585-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.332 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:39.332 INFO 1433467 --- [ForkJoinPool-1386585-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.332 INFO 1433467 --- [ForkJoinPool-1386585-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.332 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:39.332 INFO 1433467 --- [ForkJoinPool-1386585-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:39.333 INFO 1433467 --- [ForkJoinPool-1386585-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:56:39.333 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:39.334 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:39.335 INFO 1433467 --- [alarm-detect-20,929ca6d209f14a06,f67b2a3ba7c9] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:56:39.335 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.336 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.336 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.336 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:39.336 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:39.339 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:56:39.341 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:39.341 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:56:39.341442683+08:00 2024-03-13 15:56:39.342 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.342 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:39.344 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.344 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:56:39.341+08:00 2024-03-13 15:56:39.350 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:56:39.351 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:56:39.352 INFO 1433467 --- [ForkJoinPool-1386585-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.344 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.352 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:56:39.352 INFO 1433467 --- [ForkJoinPool-1386585-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.352 INFO 1433467 --- [ForkJoinPool-1386585-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.352 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:56:39.352638255+08:00 2024-03-13 15:56:39.353 INFO 1433467 --- [ForkJoinPool-1386585-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:39.353 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:56:39.353405861+08:00 2024-03-13 15:56:39.354 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:39.354 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:56:39.341+08:00 and status AVAILABLE 2024-03-13 15:56:39.354 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:56:39.353+08:00 2024-03-13 15:56:39.354 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:56:39.353+08:00 2024-03-13 15:56:39.354 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:56:39.355 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:56:39.354588390+08:00 2024-03-13 15:56:39.355 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:56:39.355 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:56:39.355+08:00 2024-03-13 15:56:39.355 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.356 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:56:39.353+08:00 and status AVAILABLE 2024-03-13 15:56:39.357 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:56:39.353+08:00 and status AVAILABLE 2024-03-13 15:56:39.357 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:56:39.356802855+08:00 2024-03-13 15:56:39.357 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.357 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:56:39.357575561+08:00 2024-03-13 15:56:39.357 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:56:39.357+08:00 2024-03-13 15:56:39.358 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:56:39.355+08:00 and status AVAILABLE 2024-03-13 15:56:39.358 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:56:39.358+08:00 2024-03-13 15:56:39.358 INFO 1433467 --- [ForkJoinPool-1386585-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:56:39.359 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.359 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:56:39.357+08:00 and status AVAILABLE 2024-03-13 15:56:39.359 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:56:39.358+08:00 and status AVAILABLE 2024-03-13 15:56:39.360 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:56:39.361 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:56:39.361 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:56:39.361 INFO 1433467 --- [pool-Check host task for agent13,d505c6875a514ae0,cb394676ad81] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:56:39.362 INFO 1433467 --- [pool-Check host task for agent10,d505c6875a514ae0,fedf33173e91] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for agent26,d505c6875a514ae0,96853a3d1b33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:56:39.363 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:56:39.364 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:56:39.364261836+08:00 2024-03-13 15:56:39.365 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.366 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:56:39.365+08:00 2024-03-13 15:56:39.366 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:56:39.366 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:56:39.366 INFO 1433467 --- [pool-Check host task for agent14,d505c6875a514ae0,1caa75dccada] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:56:39.367 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:39.367 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:56:39.365+08:00 and status AVAILABLE 2024-03-13 15:56:39.368 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:39.368 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.369 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:56:39.369 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:56:39.369 INFO 1433467 --- [pool-Check host task for agent12,d505c6875a514ae0,952e43a6eea7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:56:39.370 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:39.370 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:56:39.371 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:56:39.371 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:56:39.371 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:56:39.371 INFO 1433467 --- [pool-Check host task for agent21,d505c6875a514ae0,356b7320f7aa] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:56:39.372 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:39.372 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:56:39.373 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:56:39.373 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff17,12c93531f5a54402,d254bee2f074] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 ERROR 1433467 --- [pool-Check host task for clockdiff17,12c93531f5a54402,d254bee2f074] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff16,12c93531f5a54402,7227dbeab2dd] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 ERROR 1433467 --- [pool-Check host task for clockdiff16,12c93531f5a54402,7227dbeab2dd] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff17,12c93531f5a54402,d254bee2f074] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff16,12c93531f5a54402,7227dbeab2dd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff12,12c93531f5a54402,7d2138e338a7] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 ERROR 1433467 --- [pool-Check host task for clockdiff12,12c93531f5a54402,7d2138e338a7] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff15,12c93531f5a54402,6babde3707ec] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.382 ERROR 1433467 --- [pool-Check host task for clockdiff15,12c93531f5a54402,6babde3707ec] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:56:39.382 WARN 1433467 --- [pool-Check host task for clockdiff15,12c93531f5a54402,6babde3707ec] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.418 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:56:39.418 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:56:39.418 INFO 1433467 --- [pool-Check host task for agent29,d505c6875a514ae0,cc01aabf6046] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:56:39.418 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:56:39.419 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:56:39.419 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:56:39.420 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:56:39.422 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:56:39.426 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:56:39.431 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:56:39.432 WARN 1433467 --- [pool-Check host task for clockdiff12,12c93531f5a54402,7d2138e338a7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.433 WARN 1433467 --- [pool-scanner-1609800914-1,3f7a74e760e344a6,75123d1e2d4c] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:56:39.433 WARN 1433467 --- [pool-Check host task for clockdiff23,12c93531f5a54402,6604970ce0df] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.433 ERROR 1433467 --- [pool-Check host task for clockdiff23,12c93531f5a54402,6604970ce0df] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:56:39.434 WARN 1433467 --- [pool-Check host task for clockdiff23,12c93531f5a54402,6604970ce0df] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.434 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.434 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.438 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.440 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.440 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.440 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.441 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:56:39.442 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.456 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.457 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.459 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:56:39.459 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.462 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.462 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.462 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:56:39.463 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.463 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.463 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:39.463 WARN 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:56:39.463 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.463 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:56:39.463 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.464 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:56:39.464 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:56:39.464 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:39.469 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:56:39.471 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:39.472 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:39.475 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.477 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:56:39.490 WARN 1433467 --- [pool-Check host task for clockdiff9,12c93531f5a54402,2adeb966527d] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.490 ERROR 1433467 --- [pool-Check host task for clockdiff9,12c93531f5a54402,2adeb966527d] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:56:39.490 WARN 1433467 --- [pool-Check host task for clockdiff9,12c93531f5a54402,2adeb966527d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:56:39.490 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:56:39.491 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:56:39.491 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:56:39.492 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:39.492 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:56:39.497 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:56:39.501 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:56:39.502 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.503 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:56:39.503 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:56:39.504 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:56:39.506 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.507 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.507 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.507 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.508 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:39.509 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:56:39.511 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:56:39.514 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:56:39.514 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.515 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:56:39.516 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:56:39.518 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.519 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.520 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:56:39.520 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.521 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.530 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:56:39.533 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:56:39.534 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:56:39.536 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:56:39.541 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:56:39.542 INFO 1433467 --- [pool-Check host task for host21,15c7610b60d249a8,5203f7ca71e0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:56:39.542 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:56:39.547 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:56:39.548 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:56:39.553 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:56:39.553 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:56:39.558 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:56:39.561 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:56:39.561 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:56:39.564 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.564 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.570 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.575 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.578 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.578 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:56:39.584 INFO 1433467 --- [pool-Check host task for host5,15c7610b60d249a8,986decb5a93c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:56:39.595 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:56:39.598 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:56:39.600 INFO 1433467 --- [pool-Check host task for host33,15c7610b60d249a8,af7cd542b023] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:56:39.604 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:56:39.606 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:56:39.613 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:56:39.616 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:56:39.617 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:56:39.618 INFO 1433467 --- [pool-Check host task for host29,15c7610b60d249a8,a2bb4971db4b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:56:39.620 INFO 1433467 --- [pool-Check host task for host12,15c7610b60d249a8,645d7d7a475c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:56:39.624 INFO 1433467 --- [pool-Check host task for host27,15c7610b60d249a8,fd4e490558bb] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:56:39.642 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:56:39.649 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.649 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.649 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.649 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.650 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.650 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:56:39.650 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.650 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:56:39.650 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.651 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:56:39.659 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:56:39.659 INFO 1433467 --- [ForkJoinPool-1386584-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316540069000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:56:39.660 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:56:39.668 INFO 1433467 --- [pool-Check host task for host55,15c7610b60d249a8,b349a742df31] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:56:39.674 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.675 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.675 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.675 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.675 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:56:39.678 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.678 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.678 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:56:39.681 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:39.687 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:39.687 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:39.696 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:39.696 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:39.708 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:39.708 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:39.723 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:39.723 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:56:39.731 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:39.731 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:56:39.740 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:56:39.752 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.753 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.753 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.753 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.754 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:56:39.758 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:56:39.761 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:56:39.765 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:56:39.767 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:56:39.789 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:56:39.822 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:56:39.839 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:56:39.840 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.840 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:56:39.840 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:56:39.840 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:56:39.858 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:56:39.858 INFO 1433467 --- [ForkJoinPool-1386584-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:56:39.912 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805859,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:40.161 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:40.162 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:41.912 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805882,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:43.162 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:43.164 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:43.913 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805904,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:45.913 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805927,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:46.108 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805930,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:56:46.164 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:46.168 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:46.336 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : Starting SQL_INDEX_DIAGNOSER, Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.346 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : SQL_INDEX_DIAGNOSER : for [Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null)] 2024-03-13 15:56:46.361 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.374 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.380 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.394 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.395 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.397 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.398 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.400 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.403 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.407 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.411 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.415 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.416 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : INEFFECTIVE_HINT found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.418 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_USED found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.420 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : TABLE_SCAN_INDEX_NOT_EXISTS found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.421 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.SqlIndexDiagnoseService : AWFUL_PERFORMANCE_INDEX_USED found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:50:46.238, end=2024-03-13T15:55:46.331} 2024-03-13 15:56:46.421 INFO 1433467 --- [pool-14-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : SQL_INDEX_DIAGNOSER, Found out 0 dubious sqls 2024-03-13 15:56:46.787 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:56:46.787588+08:00, lastProcessTime=2024-03-13T15:56:36.749831+08:00 2024-03-13 15:56:46.901 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:56:46.787588+08:00 2024-03-13 15:56:46.986 INFO 1433467 --- [pool-ocp-schedules-9,09b88fa0474144a5,fb0e7e0c1e79] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:56:47.908 INFO 1433467 --- [pool-clean-time-series-data15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:56:47.914 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805951,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:47.925 INFO 1433467 --- [pool-print-time-series-interval15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316607: {1710316500=1394, 1710316515=1394, 1710316530=1394, 1710316545=1394, 1710316560=1394, 1710316575=1394, 1710316590=1394, 1710316605=60}, tsDataMapSize: 1122360, readyDataCacheSize: 8288 2024-03-13 15:56:47.949 INFO 1433467 --- [pool-clean-time-series-data60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:56:47.949 INFO 1433467 --- [pool-print-time-series-interval60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316607: {1710316440=32, 1710316500=34, 1710316560=6}, tsDataMapSize: 17768, readyDataCacheSize: 18488 2024-03-13 15:56:48.934 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,ed8449c482c043d0,c10253509d89] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.148, traceId=ed8449c482c043d0, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:56:48.950 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,ed8449c482c043d0,c10253509d89] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=ed8449c482c043d0, duration=17 ms] 2024-03-13 15:56:49.169 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:49.170 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:49.273 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805967,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:56:49.321 WARN 1433467 --- [pool-scanner-1609801501-1,055b165240814579,e1a24e61fc08] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:56:49.408 INFO 1433467 --- [alarm-detect-5,97358a24c1b245f3,f406dfe64c04] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:56:49.410 INFO 1433467 --- [alarm-detect-5,97358a24c1b245f3,f406dfe64c04] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:56:49.438 INFO 1433467 --- [alarm-detect-9,b39435cebda5432f,587201d44243] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316575 ~ 1710316599 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316590=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:56:49.438 INFO 1433467 --- [alarm-detect-9,b39435cebda5432f,587201d44243] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316575 ~ 1710316599 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316590=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:56:49.438 INFO 1433467 --- [alarm-detect-9,b39435cebda5432f,587201d44243] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316575 ~ 1710316599 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316590=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:56:49.439 INFO 1433467 --- [alarm-detect-9,b39435cebda5432f,587201d44243] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316575 ~ 1710316599 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316590=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:56:49.439 INFO 1433467 --- [alarm-detect-9,b39435cebda5432f,587201d44243] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316575 ~ 1710316599 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316590=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:56:49.439 INFO 1433467 --- [alarm-detect-9,b39435cebda5432f,587201d44243] c.o.o.a.core.detect.AlarmRuleDetector : detectByMetric done, alarmType=host_ntp_service_not_exist, needSendingCount=2, detectAlarmsCount=2, matchedTargets=[alarm_template_id=3:host=10.10.10.148-2, alarm_template_id=3:host=10.10.10.147-2, alarm_template_id=3:host=10.10.10.146-2, alarm_template_id=3:host=10.10.10.144-2, alarm_template_id=3:host=10.10.10.143-2] 2024-03-13 15:56:49.914 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805975,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:51.914 INFO 1433467 --- [SimpleAsyncTaskExecutor-89805998,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:52.170 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:52.171 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:53.210 INFO 1433467 --- [pool-ocp-schedules-13,22dfcaa697da480d,fa903a3f4443] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:56:53.223 INFO 1433467 --- [pool-ocp-schedules-13,22dfcaa697da480d,fa903a3f4443] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:56:53.223 INFO 1433467 --- [pool-ocp-schedules-13,22dfcaa697da480d,fa903a3f4443] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:56:53.915 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806021,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:55.171 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:55.172 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:55.724 INFO 1433467 --- [pool-ocp-schedules-11,1ac182377c3648ce,e43c3acf8eba] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2024-03-13 15:56:55.724 INFO 1433467 --- [pool-ocp-schedules-11,1ac182377c3648ce,e43c3acf8eba] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=609459010050985984 2024-03-13 15:56:55.728 INFO 1433467 --- [pool-ocp-schedules-11,1ac182377c3648ce,e43c3acf8eba] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2024-03-13 15:56:55.729 INFO 1433467 --- [pool-ocp-schedules-11,1ac182377c3648ce,e43c3acf8eba] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624101396131614721 2024-03-13 15:56:55.733 INFO 1433467 --- [pool-ocp-schedules-11,1ac182377c3648ce,e43c3acf8eba] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2024-03-13 15:56:55.733 INFO 1433467 --- [pool-ocp-schedules-11,1ac182377c3648ce,e43c3acf8eba] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624425259050733570 2024-03-13 15:56:55.915 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806044,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:56.108 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806047,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:56:56.449 INFO 1433467 --- [pool-Check obproxy10,93f7de159c324826,a263344a0a2c] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:56:56.455 INFO 1433467 --- [pool-Check obproxy8,93f7de159c324826,1b613c79b2e1] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:56:56.457 INFO 1433467 --- [pool-Check obproxy12,93f7de159c324826,2c72b1b0a059] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:56:56.483 INFO 1433467 --- [pool-Check obproxy5,93f7de159c324826,8893871f3ae6] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:56:56.747 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:56:56.747366+08:00, lastProcessTime=2024-03-13T15:56:46.787588+08:00 2024-03-13 15:56:56.874 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:56:56.747366+08:00 2024-03-13 15:56:56.999 INFO 1433467 --- [pool-ocp-schedules-3,912fc618f83c4b81,dcb21cdd2bc8] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:56:57.916 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806068,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:56:58.172 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:56:58.173 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:56:59.100 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,c01a7250b12e4d36,97326f43eff7] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.143, traceId=c01a7250b12e4d36, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:56:59.106 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,c01a7250b12e4d36,97326f43eff7] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.143, traceId=c01a7250b12e4d36, duration=7 ms] 2024-03-13 15:56:59.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806083,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:56:59.282 INFO 1433467 --- [alarm-detect-16,aca906aba1d24c13,faa90b389f83] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:56:59.282 INFO 1433467 --- [alarm-detect-10,11edd1326afd43e8,ea34bc7c1547] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:56:59.283 INFO 1433467 --- [alarm-detect-9,c15418cbc4ad4687,f62a80dd63e4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:56:59.287 INFO 1433467 --- [alarm-detect-10,11edd1326afd43e8,ea34bc7c1547] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:56:59.288 INFO 1433467 --- [alarm-detect-16,aca906aba1d24c13,faa90b389f83] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:56:59.288 INFO 1433467 --- [alarm-detect-9,c15418cbc4ad4687,f62a80dd63e4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:56:59.311 WARN 1433467 --- [pool-scanner-1609802125-1,e364c0d7078a403f,287b6b139973] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:56:59.422 INFO 1433467 --- [alarm-detect-10,5cb39963d8994309,25ee77c3696c] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316590 ~ 1710316609 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316605=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:56:59.422 INFO 1433467 --- [alarm-detect-10,5cb39963d8994309,25ee77c3696c] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316590 ~ 1710316609 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316605=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:56:59.422 INFO 1433467 --- [alarm-detect-10,5cb39963d8994309,25ee77c3696c] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316590 ~ 1710316609 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316605=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:56:59.422 INFO 1433467 --- [alarm-detect-10,5cb39963d8994309,25ee77c3696c] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316590 ~ 1710316609 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316605=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:56:59.422 INFO 1433467 --- [alarm-detect-10,5cb39963d8994309,25ee77c3696c] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316590 ~ 1710316609 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316605=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:56:59.422 INFO 1433467 --- [alarm-detect-10,5cb39963d8994309,25ee77c3696c] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316590 ~ 1710316609 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316605=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:56:59.422 INFO 1433467 --- [alarm-detect-10,5cb39963d8994309,25ee77c3696c] c.o.o.a.core.detect.AlarmRuleDetector : detectByMetric done, alarmType=host_ntp_service_not_exist, needSendingCount=1, detectAlarmsCount=1, matchedTargets=[alarm_template_id=3:host=10.10.10.148-2, alarm_template_id=3:host=10.10.10.147-2, alarm_template_id=3:host=10.10.10.146-2, alarm_template_id=3:host=10.10.10.145-2, alarm_template_id=3:host=10.10.10.144-2, alarm_template_id=3:host=10.10.10.143-2] 2024-03-13 15:56:59.916 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806092,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:00.000 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.s.task.scheduled.OcpSchedules : Refresh OCP properties. 2024-03-13 15:57:00.000 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.ocp.core.property.PropertyService : Refresh config properties 2024-03-13 15:57:00.009 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : new: [] 2024-03-13 15:57:00.009 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : init spring 2024-03-13 15:57:00.009 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : have already run initialize. 2024-03-13 15:57:00.010 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : starting 2024-03-13 15:57:00.016 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : new: [] 2024-03-13 15:57:00.016 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : init spring 2024-03-13 15:57:00.016 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : have already run initialize. 2024-03-13 15:57:00.016 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : starting 2024-03-13 15:57:00.039 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : environmentPrepared 2024-03-13 15:57:00.044 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : contextPrepared 2024-03-13 15:57:00.044 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : set internal properties as the first PropertySource 2024-03-13 15:57:00.044 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : register bean ProgressHandler 2024-03-13 15:57:00.044 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : set internal property ocp.version to 4.2.1 2024-03-13 15:57:00.044 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : ocp.version in spring: 4.2.1 2024-03-13 15:57:00.050 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : contextLoaded 2024-03-13 15:57:00.425 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.DataSourceInterceptor : dataSource dataSource already initialized. skipped. class: org.springframework.jdbc.datasource.DriverManagerDataSource 2024-03-13 15:57:00.566 INFO 1433467 --- [pool-prometheus-minute-schedule-1,184947afb1954613,2d4fa0ffec69] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1710316680, interval=60 2024-03-13 15:57:00.567 INFO 1433467 --- [pool-prometheus-cache-monitor1,8e77e72a6da54919,a0107771de8c] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":12075,"minuteWriteQueue":0,"minuteReadCacheSize":12328,"secondWriteCacheSize":3324,"seriesIdKeySize":67930,"secondWriteQueue":0,"minuteWriteCacheSize":12401} 2024-03-13 15:57:00.567 INFO 1433467 --- [pool-prometheus-cache-monitor1,8e77e72a6da54919,a0107771de8c] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{go_memory_classes_heap_stacks_bytes=15, ob_sysstat_40012=19, ob_server_resource_memory_assigned_bytes=13, ob_sysstat_40011=19, ob_sysstat_40010=19, ob_unit_num=13, ob_database_disk_required_size=35, go_memstats_mspan_sys_bytes=15, go_memstats_stack_sys_bytes=15, ob_tenant_request_queue_total_size=18, jvm_gc_memory_allocated_bytes_total=3, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=18, ob_sysstat_40019=19, observer_fd_count=13, ob_sysstat_40018=19, processor_process_metrics_total=21, jvm_gc_max_data_size_bytes=3, go_memstats_frees_total=15, sql_audit_input_collect_data_jumped_total=8, host_available=64, mysql_output_write_sql_failed_count=17, ob_database_disk_data_size=34, ob_system_event_detail_total_timeouts=763, ob_tenant_memstore_freeze_cnt=14, ob_latch_sleep_total=312, ob_tenant_unit_min_memory=25, ob_sysstat_60087=19, ob_memstore_total_bytes=20, metric_cardinality_count=12, process_resident_memory_bytes=15, jvm_threads_states_threads=8, ob_cache_size_bytes=34, ob_tenant_unit_max_memory=25, input_collect_metrics_total=26, ob_sysstat_60083=19, ob_sysstat_50039=19, ob_sysstat_50038=19, node_disk_io_time_weighted_seconds_total=33, process_open_fds=15, ob_sysstat_50037=19, process_memory_rss_bytes=33, ob_server_resource_cpu_assigned_percent=13, ob_sysstat_50030=19, go_memory_classes_profiling_buckets_bytes=15, go_memstats_heap_inuse_bytes=15, ob_sysstat_40118=19, ob_sysstat_40117=19, ob_sysstat_40116=19, node_memory_memfree_bytes=26, ob_sysstat_40005=19, process_exists=43, ob_sysstat_40004=19, ob_sysstat_40003=19, ob_sysstat_40002=19, ob_sysstat_40001=19, go_gc_duration_seconds=20, go_memory_classes_metadata_mspan_inuse_bytes=15, ob_server_resource_disk_bytes=13, ob_sysstat_40000=19, odp_current_session=26, process_thread_count=33, ob_sysstat_50040=19, ob_sysstat_40009=19, ob_sysstat_40008=19, ob_disk_free_bytes=13, node_filesystem_avail_bytes=72, ob_sysstat_40007=19, ob_sysstat_40006=19, ob_backup_storage_capacity_bytes=650, ob_unit_config_max_iops=21, ob_plan_cache_memory_bytes=18, node_network_transmit_errs_total=28, go_gc_duration_seconds_count=15, monagent_oceanbase_plan_explain_size=4, ob_sysstat_50011=19, ob_sysstat_50010=19, node_disk_read_bytes_total=33, ob_compaction_is_suspended=19, mysql_output_write_sql_milliseconds_summary_count=17, go_memstats_lookups_total=15, mysql_output_write_sql_count=17, process_cpu_percent=33, ob_sysstat_60057=19, host_agent_process_status=18, ob_server_resource_cpu=13, ob_tenant_server_data_index_row_count=20, go_memstats_alloc_bytes=15, go_sched_gomaxprocs_threads=15, node_cpu_seconds_total_native=38, ob_sysstat_140002=19, ob_sysstat_140005=19, ob_sysstat_140006=19, ob_sysstat_140003=19, ob_tenant_resource_min_memory=18, ob_memstore_freeze_times=20, ob_sysstat_50022=19, jvm_gc_pause_seconds_count=8, ob_sysstat_50021=19, jvm_gc_overhead_percent=3, go_memory_classes_heap_unused_bytes=15, ob_cluster_status_check=13, jvm_memory_committed_bytes=10, node_net_bandwidth_bps=29, monagent_oceanbase_db_pool_idle=4, ob_tenant_server_data_index_data_size=20, go_gc_heap_allocs_by_size_bytes_total_count=15, go_memstats_heap_idle_bytes=15, go_gc_cycles_forced_gc_cycles_total=15, go_gc_pauses_seconds_total_bucket=26, ob_server_resource_memory_bytes=13, ob_tenant_assigned_mem_assigned=18, ob_tenant_memstore_minor_freeze_times=14, ob_tenant_server_required_size=22, ob_sysstat_60031=19, process_fd_count=33, ob_tenant_unit_max_cpu=25, ob_latch_wait_seconds_total=312, go_memory_classes_metadata_mcache_free_bytes=15, node_memory_buffers_bytes=26, ob_sysstat_50009=19, go_memstats_heap_released_bytes=15, node_network_receive_bytes_total=28, ob_sysstat_50005=19, ob_sysstat_50008=19, ob_sysstat_50001=19, ob_sysstat_50004=19, exposer_request_latencies_count=6, ob_sysstat_50000=19, ob_session_active_num=18, mysql_output_context_timeout_discard_metrics_count=9, go_memory_classes_metadata_mcache_inuse_bytes=15, node_disk_read_time_seconds_total=33, ob_sysstat_60021=19, ob_sysstat_60020=19, ob_sysstat_60024=19, ob_compaction_frozen_version=19, ob_sysstat_60023=19, metric_count=15, ob_sysstat_60022=19, obproxysys_connectable=1556, ob_tenant_assigned_cpu_assigned=18, mysql_output_write_sql_retry_count=17, ob_tenant_unit_min_cpu=25, ob_unit_config_max_cpu=21, http_request_milliseconds_summary_count=75, monagent_oceanbase_audit_timeinfo_cache_size=4, http_request_milliseconds_summary=78, ob_latch_immediate_miss_total=312, ob_latch_spin_get_total=312, ob_log_level_count=17, monagent_pipeline_report_metrics_total=16, exposer_transferred_bytes_total=6, ob_tenant_context_memory_count=841, node_disk_reads_completed_total=33, ob_sysstat_60019=19, ob_sysstat_60003=19, ob_sysstat_60002=19, ob_sysstat_60001=19, ob_sysstat_60000=19, ob_latch_miss_total=312, go_sched_latencies_seconds_count=15, ocp_meta_db_health=5, ob_disk_total_bytes=13, go_memstats_heap_objects=15, go_memstats_mcache_inuse_bytes=15, ob_sysstat_60005=19, ob_sysstat_60004=19, go_memory_classes_other_bytes=15, ob_memstore_active_bytes=20, go_memstats_alloc_bytes_total=15, ob_tenant_server_partition_replica_count=21, process_count=34, ob_system_event_detail_total_waits=763, odp_used_connections=25, backup_storage_capacity_used_percent=8, go_gc_heap_frees_objects_total=15, go_memory_classes_metadata_mspan_free_bytes=15, ob_disk_log_used_bytes=13, ob_compaction_is_error=19, go_gc_limiter_last_enabled_gc_cycle=15, monagent_pipeline_execute_total=24, node_filesystem_files=86, process_start_time_seconds=15, go_memstats_heap_alloc_bytes=15, ob_tenant_resource_max_cpu=18, ntp_service_available=26, go_memstats_mspan_inuse_bytes=15, mysql_output_write_sql_milliseconds_summary=20, go_gc_heap_frees_by_size_bytes_total_count=15, monagent_pipeline_execute_seconds_total=24, ob_sysstat_80001=19, go_memstats_sys_bytes=15, go_memory_classes_heap_free_bytes=15, system_load_average_1m=3, monagent_oceanbase_audit_stat_cache_size=4, ob_tenant_memory_used_bytes=22, obproxy_server_max_connections=7, ob_sysstat_80002=19, go_memstats_gc_cpu_fraction=15, ob_sysstat_80003=19, monagent_oceanbase_audit_cache_size=4, ob_tenant_context_memory_used_bytes=841, ob_sysstat_80007=19, node_disk_writes_completed_total=33, ob_sysstat_80008=19, ob_sysstat_80009=19, go_sched_goroutines_goroutines=15, go_memory_classes_metadata_other_bytes=15, go_gc_cycles_total_gc_cycles_total=15, system_cpu_usage=3, process_cpu_usage=3, node_disk_written_bytes_total=33, monagent_oceanbase_ash_cache_size=4, go_goroutines=15, ob_sysstat_80023=19, exposer_request_latencies_sum=6, ob_unit_config_min_cpu=21, process_virtual_memory_max_bytes=15, go_gc_heap_frees_bytes_total=15, ob_tenant500_memory_hold_bytes=13, exposer_scrapes_total=6, go_gc_heap_allocs_by_size_bytes_total_bucket=27, ob_sysstat_80025=19, ob_sysstat_80026=19, oceanbase_connectivity=15, node_memory_sreclaimable_bytes=26, go_sched_latencies_seconds_sum=15, tcp_retrans=25, exporter_export_metrics_bytes_total=26, node_filesystem_size_bytes=72, http_server_requests_seconds_count=203, ob_compaction_frozen_time=19, monagent_input_sql_duration_seconds_sum=55, log_stream_degraded_count=18, ob_plan_cache_hit_total=18, node_filesystem_readonly=72, ob_sysstat_80040=19, ob_sysstat_80041=19, io_await=52, go_gc_heap_allocs_objects_total=15, go_memstats_mallocs_total=15, ob_unit_config_max_memory_bytes=21, go_gc_stack_starting_size_bytes=15, io_util=52, ob_sysstat_200002=19, ob_sysstat_200001=19, ob_tenant_disk_v4_data_size=18, go_sched_latencies_seconds_bucket=26, ob_tenant_assigned_cpu_total=18, ob_tenant_server_available_disk_free=19, node_disk_write_time_seconds_total=33, cpu_count=25, output_write_metrics_total=9, ob_cluster_sync=15, go_gc_heap_objects_objects=15, odp_transaction_total=16, ob_system_event_detail_time_waited_us=763, process_max_fds=15, http_status_count=20, ob_sysstat_80057=19, ocp_alarm_detect_duration_ms_count=115, go_gc_heap_allocs_by_size_bytes_total_sum=15, go_memstats_mcache_sys_bytes=15, ob_tenant_disk_v4_log_size=18, go_cgo_go_to_c_calls_calls_total=15, ocp_alarm_detect_duration_ms_sum=115, jvm_threads_live_threads=3, node_memory_cached_bytes=26, ob_table_num=16, ob_sysstat_10000=19, ob_sysstat_10006=19, ob_sysstat_10005=19, ob_sysstat_10002=19, ob_sysstat_10001=19, go_memstats_last_gc_time_seconds=15, ob_sysstat_10004=19, ob_sysstat_10003=19, monitor_exporter_avaliable=62, log_tailer_count=15, ob_server_num=25, ob_session_all_num=18, ob_system_event_time_waited=24, ob_tenant_resource_min_cpu=18, monagent_plugin_execute_seconds_total=39, go_gc_cycles_automatic_gc_cycles_total=15, go_gc_heap_allocs_bytes_total=15, go_memstats_gc_sys_bytes=15, ob_latch_get_total=312, go_info=16, log_tailer_processed_file_count=21, node_load15=24, ob_disk_log_assigned_bytes=13, go_memory_classes_heap_released_bytes=15, jvm_memory_max_bytes=10, ob_unit_config_min_iops=21, ob_sysstat_30080=19, ob_sysstat_30082=19, ob_sysstat_30081=19, ob_sysstat_30084=19, ob_sysstat_30083=19, ob_sysstat_30086=19, ob_sysstat_30085=19, ob_tenant_log_disk_total_bytes=18, ob_memstore_freeze_trigger_bytes=20, ob_waitevent_wait_seconds_total=18, ob_server_stopped_duration_seconds=15, go_memstats_next_gc_bytes=15, node_filesystem_files_free=86, ob_memtable_max_snapshot_duration_seconds=18, monagent_oceanbase_audit_unique_key_cache_size=4, process_virtual_memory_bytes=15, ob_sysstat_20002=19, net_connectivity=15, ob_sysstat_20001=19, monagent_input_sql_duration_seconds_bucket=65, ob_latch_immediate_get_total=312, node_cpu_seconds_total=32, ob_sysstat_20000=19, ob_index_error_num=13, ob_tenant_server_available_disk_total=19, ob_tenant_memory_hold_bytes=22, ob_sysstat_30066=19, go_memstats_buck_hash_sys_bytes=15, mysql_output_write_sql_milliseconds_summary_sum=17, ob_tenant_disk_total_size=18, odp_sql_request_total=53, mysql_output_write_task_size=17, exporter_export_metrics_total=26, ob_disk_log_total_bytes=13, system_cpu_count=3, ob_sysstat_130004=19, process_uptime_seconds=33, node_ntp_offset_seconds=25, go_memstats_heap_sys_bytes=15, ob_sysstat_140012=19, ob_sysstat_140013=19, jvm_gc_pause_seconds_max=8, ob_tenant_request_queue_large_size=18, ob_sysstat_130000=19, go_memstats_other_sys_bytes=15, ob_sysstat_130001=19, jvm_memory_used_bytes=10, ob_sysstat_130002=19, go_threads=15, http_request_milliseconds_summary_sum=75, http_server_requests_seconds_sum=203, ob_tenant_assigned_mem_total=18, node_memory_memtotal_bytes=26, node_network_transmit_drop_total=28, go_gc_pauses_seconds_total_count=15, exposer_request_latencies=9, ob_waitevent_wait_total=18, go_gc_heap_goal_bytes=15, ob_tenant_log_disk_used_bytes=18, jvm_gc_memory_promoted_bytes_total=3, odp_sql_cost_total=51, ob_sysstat_30011=19, ob_sysstat_30010=19, ob_sysstat_30013=19, ob_sysstat_30012=19, ob_tenant500_memory_used_bytes=13, node_network_transmit_bytes_total=28, node_network_receive_drop_total=28, go_gc_heap_tiny_allocs_objects_total=15, go_gc_duration_seconds_sum=15, ob_server_resource_memory_assigned_percent=13, jvm_gc_pause_seconds_sum=8, go_memory_classes_heap_objects_bytes=15, node_network_receive_errs_total=28, observer_ulimit_max_fd_count=13, ob_plan_cache_access_total=18, ob_tenant_server_row_count=22, monagent_oceanbase_plan_cache_size=4, ob_tenant_server_data_size=22, monagent_oceanbase_audit_sample_cache_size=4, go_gc_heap_frees_by_size_bytes_total_sum=15, ob_sysstat_40030=19, node_load1=24, process_cpu_seconds_total=15, ob_sysstat_120001=19, go_memory_classes_total_bytes=15, ob_sysstat_120002=19, ob_sysstat_120000=19, ob_tenant_resource_max_memory=18, ob_sysstat_120009=19, go_memstats_stack_inuse_bytes=15, ob_sysstat_120008=19, node_load5=24, ob_sysstat_120005=19, go_memory_classes_os_stacks_bytes=15, ob_sysstat_120006=19, jvm_threads_peak_threads=3, monagent_input_sql_duration_seconds_count=55, ob_tenant_context_memory_hold_bytes=841, obproxy_memory_limit_bytes=7, ob_compaction_last_version=19, ob_sysstat_30000=19, ob_server_resource_cpu_assigned=13, ob_sysstat_30002=19, ob_sysstat_30001=19, monagent_oceanbase_plan_timeinfo_cache_size=4, monagent_plugin_execute_total=39, ob_sysstat_30006=19, ob_sysstat_30005=19, ob_sysstat_30008=19, ob_sysstat_30007=19, ob_sysstat_30009=19, go_gc_heap_frees_by_size_bytes_total_bucket=27, obproxy_client_max_connections=7, ob_compaction_current_timestamp=19, ob_clog_max_sync_delay_seconds=20, go_gc_pauses_seconds_total_sum=15, ob_compaction_start_time=19} 2024-03-13 15:57:00.577 INFO 1433467 --- [pool-prometheus-minute-schedule-1,184947afb1954613,2d4fa0ffec69] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=18, collectTaskCount=18 2024-03-13 15:57:00.577 INFO 1433467 --- [pool-prometheus-minute-schedule-1,184947afb1954613,2d4fa0ffec69] c.o.ocp.monitor.OcpMonitorManager : Schedule interval metric too slow, from=1710316620, count=18, elapsed=1708606303957 2024-03-13 15:57:00.699 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:01.0, nextRun=2024-03-13 15:57:00.0, currentDate=Wed Mar 13 15:57:00 CST 2024 2024-03-13 15:57:00.699 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=38, name=Detect alarms on schedule, templateName=BackupScheduleAlarmTemplateFactory, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:56:01.0, nextRunTime=2024-03-13 15:57:00.0} 2024-03-13 15:57:00.703 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Detect alarms, nodeCount=1, taskCount=1 2024-03-13 15:57:00.703 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:57:00.703 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Detect alarms, taskDefinitionId=38, state=RUNNING, operation=EXECUTE} 2024-03-13 15:57:00.717 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:01.0, nextRun=2024-03-13 15:57:00.0, currentDate=Wed Mar 13 15:57:00 CST 2024 2024-03-13 15:57:00.717 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999981, name=Collect all tenants major compactions, templateName=CollectTenantCompactionSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:56:01.0, nextRunTime=2024-03-13 15:57:00.0} 2024-03-13 15:57:00.719 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all tenants major compactions, nodeCount=1, taskCount=1 2024-03-13 15:57:00.719 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:57:00.719 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all tenants major compactions, taskDefinitionId=999981, state=RUNNING, operation=EXECUTE} 2024-03-13 15:57:00.724 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:01.0, nextRun=2024-03-13 15:57:00.0, currentDate=Wed Mar 13 15:57:00 CST 2024 2024-03-13 15:57:00.724 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999982, name=Collect SlowSql to SqlAggregateData for alarm, templateName=CollectSlowSqlSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:56:01.0, nextRunTime=2024-03-13 15:57:00.0} 2024-03-13 15:57:00.734 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect SlowSql to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:57:00.734 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:57:00.734 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect SlowSql to SqlAggregateData for alarm, taskDefinitionId=999982, state=RUNNING, operation=EXECUTE} 2024-03-13 15:57:00.754 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:01.0, nextRun=2024-03-13 15:57:00.0, currentDate=Wed Mar 13 15:57:00 CST 2024 2024-03-13 15:57:00.755 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999983, name=Collect trans stat to SqlAggregateData for alarm, templateName=CollectTransSatatSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:56:01.0, nextRunTime=2024-03-13 15:57:00.0} 2024-03-13 15:57:00.759 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect trans stat to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:57:00.759 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:57:00.759 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect trans stat to SqlAggregateData for alarm, taskDefinitionId=999983, state=RUNNING, operation=EXECUTE} 2024-03-13 15:57:00.764 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:01.0, nextRun=2024-03-13 15:57:00.0, currentDate=Wed Mar 13 15:57:00 CST 2024 2024-03-13 15:57:00.764 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999984, name=Get agent alarms, templateName=GetAgentAlarmsSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:56:01.0, nextRunTime=2024-03-13 15:57:00.0} 2024-03-13 15:57:00.766 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Get agent alarms, nodeCount=1, taskCount=1 2024-03-13 15:57:00.766 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:57:00.766 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Get agent alarms, taskDefinitionId=999984, state=RUNNING, operation=EXECUTE} 2024-03-13 15:57:00.778 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:31.0, nextRun=2024-03-13 15:57:00.0, currentDate=Wed Mar 13 15:57:00 CST 2024 2024-03-13 15:57:00.778 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:56:31.0, nextRunTime=2024-03-13 15:57:00.0} 2024-03-13 15:57:00.780 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:57:00.780 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:57:00.780 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:57:00.798 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : ready. time taken: PT0.781685329S 2024-03-13 15:57:00.798 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : environmentPrepared 2024-03-13 15:57:00.861 INFO 1433467 --- [pool-ocp-schedules-8,,] o.s.c.c.s.e.NativeEnvironmentRepository : Adding property source: Config resource 'class path resource [application.properties]' via location 'classpath:/' 2024-03-13 15:57:00.898 INFO 1433467 --- [pool-ocp-schedules-8,,] b.c.PropertySourceBootstrapConfiguration : Located property source: [BootstrapPropertySource {name='bootstrapProperties-classpath:/application.properties'}, BootstrapPropertySource {name='bootstrapProperties-ocp-default'}] 2024-03-13 15:57:00.934 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : contextPrepared 2024-03-13 15:57:00.935 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : set internal properties as the first PropertySource 2024-03-13 15:57:00.935 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : register bean ProgressHandler 2024-03-13 15:57:00.935 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : set internal property ocp.version to 4.2.1 2024-03-13 15:57:00.935 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : ocp.version in spring: 4.2.1 2024-03-13 15:57:00.935 INFO 1433467 --- [pool-ocp-schedules-8,,] o.s.boot.SpringApplication : The following 1 profile is active: "composite" 2024-03-13 15:57:00.942 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : contextLoaded 2024-03-13 15:57:01.050 INFO 1433467 --- [pool-ocp-schedules-8,,] o.s.boot.SpringApplication : Started application in 1.04 seconds (JVM running for 7707386.26) 2024-03-13 15:57:01.050 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.BootstrapRunListener : ready. time taken: PT1.040986189S 2024-03-13 15:57:01.157 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.s.c.c.w.MetadbDataSourceWrapper : [metadb] driver class=com.oceanbase.jdbc.Driver 2024-03-13 15:57:01.157 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.s.c.c.w.MetadbDataSourceWrapper : [metadb] username=meta_user@ocp 2024-03-13 15:57:01.157 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.s.c.c.w.MetadbDataSourceWrapper : [metadb] jdbc url=jdbc:oceanbase://10.10.10.148:2881/meta_database?useUnicode=true&characterEncoding=UTF8&connectTimeout=2000&socketTimeout=30000&rewriteBatchedStatements=true&zeroDateTimeBehavior=convertToNull 2024-03-13 15:57:01.157 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.o.b.spring.DataSourceInterceptor : dataSource metaDataSource already initialized. skipped. class: com.oceanbase.ocp.server.common.config.wrapper.MetadbDataSourceWrapper 2024-03-13 15:57:01.173 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:01.196 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:01.239 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016760, operation:EXECUTE, state: RUNNING 2024-03-13 15:57:01.251 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016761, operation:EXECUTE, state: RUNNING 2024-03-13 15:57:01.258 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,2c8997c23feb] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016760, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014408, task_operation=execute, latest_execution_start_time=2024-03-13T15:57:01.220+08:00, sub_task_instance_name=Collect all tenants major compactions, sub_task_instance_id=10016760}, listMap={}}, executor=10.10.10.148 2024-03-13 15:57:01.258 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,2c8997c23feb] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2024-03-13 15:57:01.259 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016762, operation:EXECUTE, state: RUNNING 2024-03-13 15:57:01.266 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016763, operation:EXECUTE, state: RUNNING 2024-03-13 15:57:01.268 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,513f5a4106f0] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016761, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014409, task_operation=execute, latest_execution_start_time=2024-03-13T15:57:01.249+08:00, sub_task_instance_name=Collect SlowSql to SqlAggregateData for alarm, sub_task_instance_id=10016761}, listMap={}}, executor=10.10.10.148 2024-03-13 15:57:01.273 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016764, operation:EXECUTE, state: RUNNING 2024-03-13 15:57:01.276 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,ce079179d60c] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016762, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014410, task_operation=execute, latest_execution_start_time=2024-03-13T15:57:01.257+08:00, sub_task_instance_name=Collect trans stat to SqlAggregateData for alarm, sub_task_instance_id=10016762}, listMap={}}, executor=10.10.10.148 2024-03-13 15:57:01.281 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,f3672353ca03] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016763, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014411, task_operation=execute, latest_execution_start_time=2024-03-13T15:57:01.264+08:00, sub_task_instance_name=Get all agent alarms, sub_task_instance_id=10016763}, listMap={}}, executor=10.10.10.148 2024-03-13 15:57:01.289 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016759, operation:EXECUTE, state: RUNNING 2024-03-13 15:57:01.292 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,49210c3e18e3] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016764, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014412, task_operation=execute, latest_execution_start_time=2024-03-13T15:57:01.271+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016764}, listMap={}}, executor=10.10.10.148 2024-03-13 15:57:01.306 INFO 1433467 --- [pool-Check obproxy9,f47a761474a84dce,1dabd9b513e2] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:57:01.306 INFO 1433467 --- [pool-Check obproxy7,f47a761474a84dce,ad96ebfb1c68] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:57:01.306 INFO 1433467 --- [pool-Check obproxy15,f47a761474a84dce,0738bf87c202] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:57:01.306 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,0d93d9922f58] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:57:01.309 INFO 1433467 --- [pool-ocp-schedules-8,,] c.o.ocp.core.property.PropertyService : Refresh config properties done 2024-03-13 15:57:01.317 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,513f5a4106f0] c.o.o.p.s.i.alarm.SlowSqlDataSource : query sqlCount of obCluster obcluster1, sql count is 0. 2024-03-13 15:57:01.317 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,513f5a4106f0] c.o.o.p.s.i.alarm.SlowSqlDataSource : query SlowSqlData of obCluster obcluster1, result is empty. 2024-03-13 15:57:01.317 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016759, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014407, task_operation=execute, latest_execution_start_time=2024-03-13T15:57:01.287+08:00, sub_task_instance_name=Detect alarms of backup, sub_task_instance_id=10016759}, listMap={}}, executor=10.10.10.148 2024-03-13 15:57:01.318 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,f41b06345f75] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:57:01.318 INFO 1433467 --- [pool-Collect all tenants major compactions6,f47a761474a84dce,4ba8cfdb5f8d] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:57:01.318 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,b41e4c5412f5] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:57:01.318 INFO 1433467 --- [pool-Collect all tenants major compactions1,f47a761474a84dce,d180fe6e6da3] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:57:01.319 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,513f5a4106f0] c.o.o.p.s.i.alarm.SlowSqlDataSource : query sqlCount of obCluster obcluster, sql count is 0. 2024-03-13 15:57:01.321 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,513f5a4106f0] c.o.o.p.s.i.alarm.SlowSqlDataSource : query SlowSqlData of obCluster obcluster, result is empty. 2024-03-13 15:57:01.325 INFO 1433467 --- [pool-subtask-executor31,f47a761474a84dce,513f5a4106f0] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016761, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:57:01.324 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,ce079179d60c] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster1, trans stat result is empty. 2024-03-13 15:57:01.325 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,ce079179d60c] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster, trans stat result is empty. 2024-03-13 15:57:01.334 INFO 1433467 --- [pool-Check obproxy9,f47a761474a84dce,1dabd9b513e2] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:57:01.335 INFO 1433467 --- [pool-Collect all tenants major compactions6,f47a761474a84dce,4ba8cfdb5f8d] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.335 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,f3672353ca03] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016763, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:57:01.336 INFO 1433467 --- [pool-Collect all tenants major compactions6,f47a761474a84dce,4ba8cfdb5f8d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.339 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,b41e4c5412f5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.339 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2024-03-13 15:57:01.339 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2024-03-13 15:57:01.339 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,b41e4c5412f5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.339 INFO 1433467 --- [pool-Check obproxy15,f47a761474a84dce,0738bf87c202] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:57:01.339 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,b41e4c5412f5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.339 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,b41e4c5412f5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.340 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,b41e4c5412f5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:57:01.340 INFO 1433467 --- [pool-Collect all tenants major compactions6,f47a761474a84dce,4ba8cfdb5f8d] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.340 INFO 1433467 --- [pool-Collect all tenants major compactions6,f47a761474a84dce,4ba8cfdb5f8d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.340 INFO 1433467 --- [pool-Collect all tenants major compactions6,f47a761474a84dce,4ba8cfdb5f8d] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:57:01.341 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,ce079179d60c] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016762, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:57:01.341 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,0d93d9922f58] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:57:01.343 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,b41e4c5412f5] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266400687471944, broadcast scn:1710266400687471944, saved frozen scn:1710266400687471944 2024-03-13 15:57:01.343 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=7 2024-03-13 15:57:01.343 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2024-03-13 15:57:01.344 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2024-03-13 15:57:01.344 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2024-03-13 15:57:01.344 INFO 1433467 --- [pool-Check obproxy7,f47a761474a84dce,ad96ebfb1c68] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:57:01.344 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:57:01.344 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,f41b06345f75] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.345 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=6 2024-03-13 15:57:01.345 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,f41b06345f75] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.345 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,f41b06345f75] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.345 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,f41b06345f75] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:57:01.346 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:57:01.347 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,f41b06345f75] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402011840869, broadcast scn:1710266402011840869, saved frozen scn:1710266402011840869 2024-03-13 15:57:01.347 INFO 1433467 --- [pool-Collect all tenants major compactions6,f47a761474a84dce,4ba8cfdb5f8d] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266403004770377, broadcast scn:1710266403004770377, saved frozen scn:1710266403004770377 2024-03-13 15:57:01.348 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:57:01.348 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.349 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:57:01.349 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:57:01.352 INFO 1433467 --- [pool-Collect all tenants major compactions1,f47a761474a84dce,d180fe6e6da3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.350 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:57:01.350 INFO 1433467 --- [pool-Collect all tenants major compactions8,f47a761474a84dce,f41b06345f75] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:57:01.351 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:57:01.350 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:57:01.352 INFO 1433467 --- [pool-Collect all tenants major compactions1,f47a761474a84dce,d180fe6e6da3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.352 INFO 1433467 --- [pool-Collect all tenants major compactions1,f47a761474a84dce,d180fe6e6da3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.352 INFO 1433467 --- [pool-Collect all tenants major compactions1,f47a761474a84dce,d180fe6e6da3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.353 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:57:01.353 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:57:01.353 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:57:01.353 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:57:01.353 INFO 1433467 --- [pool-Collect all tenants major compactions1,f47a761474a84dce,d180fe6e6da3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:57:01.353 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:57:01.356 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:57:01.356 INFO 1433467 --- [pool-Collect all tenants major compactions1,f47a761474a84dce,d180fe6e6da3] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402269282986, broadcast scn:1710266402269282986, saved frozen scn:1710266402269282986 2024-03-13 15:57:01.356 INFO 1433467 --- [pool-Collect all tenants major compactions1,f47a761474a84dce,d180fe6e6da3] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:57:01.358 INFO 1433467 --- [pool-Collect all tenants major compactions3,f47a761474a84dce,b41e4c5412f5] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:57:01.359 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.361 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.360 INFO 1433467 --- [pool-Collect all tenants major compactions6,f47a761474a84dce,4ba8cfdb5f8d] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:57:01.366 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:57:01.368 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,2c8997c23feb] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2024-03-13 15:57:01.374 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,2c8997c23feb] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016760, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:57:01.375 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,49210c3e18e3] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016764, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:57:01.419 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.421 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.423 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.425 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.428 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:57:01.432 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62889/api/v1/log/alarms, params:null 2024-03-13 15:57:01.433 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62889/api/v1/log/alarms, params:null 2024-03-13 15:57:01.433 INFO 1433467 --- [ForkJoinPool-15-worker-1,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62889/api/v1/log/alarms, params:null 2024-03-13 15:57:01.435 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62889/api/v1/log/alarms, params:null 2024-03-13 15:57:01.441 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62889/api/v1/log/alarms, params:null 2024-03-13 15:57:01.442 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62889/api/v1/log/alarms, params:null 2024-03-13 15:57:01.443 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62889/api/v1/log/alarms, params:null 2024-03-13 15:57:01.453 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:57:01.477 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.481 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.485 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.489 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:57:01.552 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.556 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.560 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.563 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.566 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:57:01.587 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:57:01.612 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.616 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.628 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.631 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.635 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.637 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.641 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:57:01.645 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:57:01.653 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.657 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.661 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [3600] 2024-03-13 15:57:01.702 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=3600, delayTasks=[] 2024-03-13 15:57:01.711 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.714 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.722 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.725 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.727 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.730 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.733 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:57:01.737 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:57:01.741 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.744 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.747 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [300] 2024-03-13 15:57:01.777 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=300, delayTasks=[] 2024-03-13 15:57:01.799 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.803 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.808 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.813 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.817 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [90] 2024-03-13 15:57:01.851 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=90, timeoutTasks=[] 2024-03-13 15:57:01.861 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.864 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.868 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.870 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.874 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [60] 2024-03-13 15:57:01.905 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=60, timeoutTasks=[] 2024-03-13 15:57:01.916 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806114,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:01.924 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.927 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.931 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:01.934 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.938 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:57:01.973 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:57:01.983 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:01.986 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.989 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:01.991 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:01.994 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:57:02.027 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:57:02.031 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster1 does not enable secondary backup 2024-03-13 15:57:02.034 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster does not enable secondary backup 2024-03-13 15:57:02.040 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackup/], metricValues size:[0]. 2024-03-13 15:57:02.043 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackupocp/], metricValues size:[0]. 2024-03-13 15:57:02.052 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,194f9ecb1ff5] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016759, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:57:02.281 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014407, status:SUCCESS 2024-03-13 15:57:02.285 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014407, status:SUCCESS 2024-03-13 15:57:02.293 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014408, status:SUCCESS 2024-03-13 15:57:02.294 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014408, status:SUCCESS 2024-03-13 15:57:02.301 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014409, status:SUCCESS 2024-03-13 15:57:02.302 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014409, status:SUCCESS 2024-03-13 15:57:02.310 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014410, status:SUCCESS 2024-03-13 15:57:02.311 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014410, status:SUCCESS 2024-03-13 15:57:02.320 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014411, status:SUCCESS 2024-03-13 15:57:02.321 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014411, status:SUCCESS 2024-03-13 15:57:02.330 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014412, status:SUCCESS 2024-03-13 15:57:02.330 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014412, status:SUCCESS 2024-03-13 15:57:03.917 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806137,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:04.196 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:04.197 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:05.917 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806160,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:06.109 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806163,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:57:06.748 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:57:06.748413+08:00, lastProcessTime=2024-03-13T15:56:56.747366+08:00 2024-03-13 15:57:06.868 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:57:06.748413+08:00 2024-03-13 15:57:07.006 INFO 1433467 --- [pool-ocp-schedules-16,8f8e21718e954fb6,244f25f31de0] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:57:07.197 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:07.198 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:07.917 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806184,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:08.116 INFO 1433467 --- [pool-15-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : Starting PLAN_CHANGE_DETECTOR, Interval{start=2024-03-13T15:53:07.863, end=2024-03-13T15:55:08.113} 2024-03-13 15:57:08.124 INFO 1433467 --- [pool-15-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR : for [Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null)] 2024-03-13 15:57:08.125 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:57:08.128 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:53:07.863, end=2024-03-13T15:55:08.113},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:57:08.128 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:57:08.128 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:57:08.136 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:57:08.137 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 2 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:53:07.863, end=2024-03-13T15:55:08.113},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:57:08.144 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:57:08.144 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:57:08.216 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230228216000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:57:08.217 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230228217000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=DD3D1866D187B8E0DC29B621473BC036, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:57:08.218 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230228218000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=E204274999A63B8CBD67199F075C5D97, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:57:08.219 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230228219000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=B7A34188E00F96CA660B2D39A3968328, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:57:08.223 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230228223000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=D7C3095058607028FFED1D7C84C3162B, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:57:08.226 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 274 2024-03-13 15:57:08.226 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:57:08.228 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 258 2024-03-13 15:57:08.228 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:57:08.228 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230228228000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, obServerId=null, obDbId=500002, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=2, obTenantId=1002, tenantName=tenant1, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=2, name=tenant1, obTenantId=1002, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:57:08.231 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 228 2024-03-13 15:57:08.231 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:57:08.231 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 261 2024-03-13 15:57:08.231 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:57:08.234 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.234 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.237 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 233 2024-03-13 15:57:08.237 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:57:08.238 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 383 2024-03-13 15:57:08.238 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:57:08.239 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.239 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=D7C3095058607028FFED1D7C84C3162B, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.240 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.240 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=E204274999A63B8CBD67199F075C5D97, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.240 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:57:08.240 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:57:08.241 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.241 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=B7A34188E00F96CA660B2D39A3968328, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.241 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:57:08.241 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:57:08.242 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:57:08.242 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:57:08.243 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:57:08.243 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:57:08.243 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.244 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=DD3D1866D187B8E0DC29B621473BC036, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.245 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:57:08.245 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:57:08.247 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.247 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.249 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 6 records,use plan union hash 2024-03-13 15:57:08.249 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.249 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=E204274999A63B8CBD67199F075C5D97, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.249 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.249 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=D7C3095058607028FFED1D7C84C3162B, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.249 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.249 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.251 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:57:08.251 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:57:08.251 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 4 records,use plan union hash 2024-03-13 15:57:08.251 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:57:08.252 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.252 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=B7A34188E00F96CA660B2D39A3968328, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.253 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.253 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=DD3D1866D187B8E0DC29B621473BC036, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.254 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:57:08.254 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 2 records,use plan union hash 2024-03-13 15:57:08.255 INFO 1433467 --- [pool-plan_change_detector4,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:57:08.218, end=2024-03-13T15:55:08.113}, clusterId=1, tenantId=2, sqlId=E204274999A63B8CBD67199F075C5D97, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:57:08.256 INFO 1433467 --- [pool-plan_change_detector7,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:57:08.223, end=2024-03-13T15:55:08.113}, clusterId=1, tenantId=2, sqlId=D7C3095058607028FFED1D7C84C3162B, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:57:08.256 INFO 1433467 --- [pool-plan_change_detector5,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:57:08.219, end=2024-03-13T15:55:08.113}, clusterId=1, tenantId=2, sqlId=B7A34188E00F96CA660B2D39A3968328, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:57:08.257 INFO 1433467 --- [pool-plan_change_detector3,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:57:08.217, end=2024-03-13T15:55:08.113}, clusterId=1, tenantId=2, sqlId=DD3D1866D187B8E0DC29B621473BC036, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:57:08.259 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 2 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:57:08.216, end=2024-03-13T15:55:08.113}, clusterId=1, tenantId=2, sqlId=5CE887DD94A690E3A5ED4EFFC6827B76, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:57:08.260 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.260 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1002, obServerId=null, obDbId=500002, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.262 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 6 records,use plan union hash 2024-03-13 15:57:08.267 INFO 1433467 --- [pool-plan_change_detector2,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 4 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:57:08.228, end=2024-03-13T15:55:08.113}, clusterId=1, tenantId=2, sqlId=0244F16B2735CEE4EABB666ECC0BE9F4, serverId=null, dbName=membership, obDbId=500002, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:57:08.267 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:57:08.268 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:53:07.863, end=2024-03-13T15:55:08.113},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:57:08.268 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:57:08.268 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:57:08.269 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql plan 2024-03-13 15:57:08.269 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : PERFORMANCE_DEGRADATION_AFTER_PLAN_CHANGED found out 2 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:53:07.863, end=2024-03-13T15:55:08.113},PlanChangePerfDegradationDetectService.Config(enabled=true, sysTenantEnabled=true, level=0, compareMin=5, cpuTimeMultiply={0=50.0, 1=20.0, 10=10.0, 100=3.0, 1000=2.0}, limit=1000, cpuTimeLimitMs=1.0, alarm=true, autoFlushPlanCache=false, minElapsedTimeMsForFlushingPlanCache=50.0) 2024-03-13 15:57:08.271 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : Found out 0 dubious sqls of Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null) 2024-03-13 15:57:08.271 INFO 1433467 --- [pool-15-thread-1,,] i.PlanChangePerfDegradationDetectService : diagnose sql stats 2024-03-13 15:57:08.343 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Query plan stat with:QueryPlanRawStatBySql(timeout=30000000, level=0, startTimeUs=1710230228343000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, obServerId=null, obDbId=201001, dynamicSql=false), context:SqlStatDataContext(clusterId=1, obClusterId=1, clusterName=obcluster1, tenantId=1, obTenantId=1, tenantName=sys, mode=MYSQL, serverId=null, obServerId=null, obVersion=4.2.1.4, startTime=null, endTime=null, parseSqlType=false, parseStatement=false, parseTable=false, obServerIds=[1, 2, 3], serverMap={1=10.10.10.145:2882, 2=10.10.10.146:2882, 3=10.10.10.147:2882}, serverIpMap={10.10.10.146=2, 10.10.10.147=3, 10.10.10.145=1}, cluster=ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING}, tenant=ObTenantEntity{id=1, name=sys, obTenantId=1, status=NORMAL}, servers=[ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}], server=null) 2024-03-13 15:57:08.360 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Top plan query ret: 789 2024-03-13 15:57:08.360 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan id 2024-03-13 15:57:08.367 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.367 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1, obServerId=null, obDbId=201001, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.369 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 0 records, use plan hash 2024-03-13 15:57:08.369 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : query plan use plan union hash 2024-03-13 15:57:08.376 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to query plan info 2024-03-13 15:57:08.376 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : queryIncrementBySql with:QueryPlan(timeout=30000000, limit=3000, startTimeUs=1710201600000000, endTimeUs=1710316508113000, obClusterId=1, clusterName=obcluster1, obTenantId=1, obServerId=null, obDbId=201001, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, sqlIdList=null, dynamicSql=false) 2024-03-13 15:57:08.379 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.i.PlanRawStatServiceImpl : Begin to map to models for 3 records,use plan union hash 2024-03-13 15:57:08.382 INFO 1433467 --- [pool-plan_change_detector1,,] c.o.o.p.s.d.i.a.PlanBindingAdvisor : Got 1 plans with param:QueryTopPlanParam(interval=Interval{start=2024-03-12T15:57:08.343, end=2024-03-13T15:55:08.113}, clusterId=1, tenantId=1, sqlId=8C447179E0DD1CCC21228D85DA5EBA8C, serverId=null, dbName=oceanbase, obDbId=201001, attachDiagnosis=false, attachPlanExplain=true, attachOutlineBindingStatus=false, dynamicSql=false) 2024-03-13 15:57:08.382 INFO 1433467 --- [pool-15-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : PLAN_CHANGE_DETECTOR, Found out 0 dubious sqls 2024-03-13 15:57:09.249 INFO 1433467 --- [pool-ocp-schedules-13,c8c7b1d85133462b,3b07fb428775] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2024-03-13 15:57:09.254 INFO 1433467 --- [pool-ocp-schedules-19,154c5394318049c7,d121bdd0df52] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:57:09.256 INFO 1433467 --- [pool-ocp-schedules-11,468352f05cc14769,856c47343b23] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:57:09.257 INFO 1433467 --- [pool-ocp-schedules-19,154c5394318049c7,d121bdd0df52] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:57:09.258 INFO 1433467 --- [pool-ocp-schedules-11,468352f05cc14769,856c47343b23] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:57:09.270 INFO 1433467 --- [pool-ocp-schedules-13,c8c7b1d85133462b,3b07fb428775] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2024-03-13 15:57:09.271 INFO 1433467 --- [pool-ocp-schedules-8,616cdeb1763044a0,9e648f1558a5] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:57:09.276 INFO 1433467 --- [pool-ocp-schedules-8,616cdeb1763044a0,9e648f1558a5] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:57:09.276 INFO 1433467 --- [pool-ocp-schedules-14,4c7fdddb937047d6,147a109744cc] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:57:09.281 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806200,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:57:09.282 INFO 1433467 --- [pool-ocp-schedules-13,e1f95931c3cf48e9,8bd8643dd3ab] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:57:09.296 INFO 1433467 --- [pool-ocp-schedules-17,14b38f165b3c4082,c8108d66acf1] c.o.o.s.task.scheduled.ClusterSchedules : Check host version schedule. 2024-03-13 15:57:09.296 INFO 1433467 --- [pool-ocp-schedules-17,14b38f165b3c4082,c8108d66acf1] o.o.c.t.e.u.DistributedExclusiveExecutor : Empty item list, taskName=Check host task for host version, skip execute 2024-03-13 15:57:09.297 INFO 1433467 --- [ForkJoinPool-1386589-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.297 INFO 1433467 --- [ForkJoinPool-1386589-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.297 INFO 1433467 --- [ForkJoinPool-1386589-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.298 INFO 1433467 --- [ForkJoinPool-1386589-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:57:09.298 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.299 INFO 1433467 --- [ForkJoinPool-1386589-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:57:09.301 INFO 1433467 --- [ForkJoinPool-1386589-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.301 INFO 1433467 --- [ForkJoinPool-1386589-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.301 INFO 1433467 --- [ForkJoinPool-1386589-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.301 INFO 1433467 --- [ForkJoinPool-1386589-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:57:09.304 INFO 1433467 --- [pool-ocp-schedules-9,44015733ab084554,3dd5290195ab] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:57:09.304 INFO 1433467 --- [pool-Check host task for clockdiff10,44015733ab084554,c737bc2ac126] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:57:09.304 INFO 1433467 --- [pool-Check host task for clockdiff5,44015733ab084554,2b002a3470d0] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:57:09.304 INFO 1433467 --- [pool-Check host task for clockdiff26,44015733ab084554,a407c7335a7e] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:57:09.304 INFO 1433467 --- [pool-Check host task for clockdiff25,44015733ab084554,2e9900b7ed83] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:57:09.305 INFO 1433467 --- [pool-Check host task for clockdiff19,44015733ab084554,c9cfdd191ed9] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:57:09.305 INFO 1433467 --- [pool-Check host task for clockdiff14,44015733ab084554,4d8fa2d64d20] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:57:09.305 INFO 1433467 --- [pool-Check host task for clockdiff11,44015733ab084554,29ccd0ca6832] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:57:09.305 INFO 1433467 --- [ForkJoinPool-1386589-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:57:09.310 INFO 1433467 --- [pool-ocp-schedules-3,1d5c408fbda04de5,920bb22701e6] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:57:09.310 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:57:09.311 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:57:09.311 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:57:09.311 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:57:09.311 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:57:09.311 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:57:09.311 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:57:09.312 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.313 INFO 1433467 --- [pool-ocp-schedules-6,3a60b1451fd548d2,298dca7c891c] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:57:09.313 INFO 1433467 --- [pool-Check host task for agent18,3a60b1451fd548d2,edd2ec1a6a0e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:57:09.313 INFO 1433467 --- [pool-Check host task for agent2,3a60b1451fd548d2,cc63146357d8] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:57:09.313 INFO 1433467 --- [pool-Check host task for agent15,3a60b1451fd548d2,76d005bcd2e3] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:57:09.313 INFO 1433467 --- [pool-Check host task for agent4,3a60b1451fd548d2,2f4c8f792487] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:57:09.313 INFO 1433467 --- [pool-Check host task for agent20,3a60b1451fd548d2,4f6b5a3bdd24] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:57:09.313 INFO 1433467 --- [pool-Check host task for agent30,3a60b1451fd548d2,d48d200057a2] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:57:09.313 INFO 1433467 --- [pool-Check host task for agent28,3a60b1451fd548d2,6f05598202a7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:57:09.314 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:57:09.315 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:57:09.315 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:57:09.317 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:57:09.317 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:57:09.317 INFO 1433467 --- [pool-Check host task for agent4,3a60b1451fd548d2,2f4c8f792487] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:57:09.318 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:57:09.318 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:57:09.318 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:57:09.318 INFO 1433467 --- [pool-Check host task for agent28,3a60b1451fd548d2,6f05598202a7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:57:09.318 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:57:09.319 INFO 1433467 --- [pool-Check host task for agent18,3a60b1451fd548d2,edd2ec1a6a0e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:57:09.319 INFO 1433467 --- [pool-Check host task for agent4,3a60b1451fd548d2,2f4c8f792487] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:57:09.319 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:57:09.319 INFO 1433467 --- [pool-Check host task for agent28,3a60b1451fd548d2,6f05598202a7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:57:09.319 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:57:09.320 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:57:09.320 INFO 1433467 --- [pool-Check host task for agent30,3a60b1451fd548d2,d48d200057a2] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:57:09.320 INFO 1433467 --- [pool-Check host task for agent18,3a60b1451fd548d2,edd2ec1a6a0e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:57:09.320 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:57:09.320 INFO 1433467 --- [pool-Check host task for agent2,3a60b1451fd548d2,cc63146357d8] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:57:09.321 INFO 1433467 --- [pool-Check host task for agent20,3a60b1451fd548d2,4f6b5a3bdd24] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:57:09.321 INFO 1433467 --- [pool-Check host task for agent15,3a60b1451fd548d2,76d005bcd2e3] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:57:09.321 INFO 1433467 --- [pool-Check host task for agent30,3a60b1451fd548d2,d48d200057a2] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:57:09.322 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:57:09.322 INFO 1433467 --- [pool-Check host task for agent20,3a60b1451fd548d2,4f6b5a3bdd24] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:57:09.322 INFO 1433467 --- [pool-Check host task for agent15,3a60b1451fd548d2,76d005bcd2e3] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:57:09.323 INFO 1433467 --- [pool-Check host task for agent2,3a60b1451fd548d2,cc63146357d8] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:57:09.334 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:57:09.336 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:57:09.336 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:57:09.336321704+08:00 2024-03-13 15:57:09.337 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:57:09.336+08:00 2024-03-13 15:57:09.338 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:57:09.337872647+08:00 2024-03-13 15:57:09.339 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:57:09.338+08:00 2024-03-13 15:57:09.342 INFO 1433467 --- [pool-Check host task for agent4,3a60b1451fd548d2,2f4c8f792487] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:57:09.342 INFO 1433467 --- [pool-Check host task for agent28,3a60b1451fd548d2,6f05598202a7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:57:09.342 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:57:09.336+08:00 and status AVAILABLE 2024-03-13 15:57:09.344 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:57:09.338+08:00 and status AVAILABLE 2024-03-13 15:57:09.344 INFO 1433467 --- [pool-Check host task for agent30,3a60b1451fd548d2,d48d200057a2] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:57:09.347 INFO 1433467 --- [pool-Check host task for agent4,3a60b1451fd548d2,2f4c8f792487] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:57:09.347 INFO 1433467 --- [pool-Check host task for agent4,3a60b1451fd548d2,2f4c8f792487] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:57:09.347 INFO 1433467 --- [pool-Check host task for agent4,3a60b1451fd548d2,2f4c8f792487] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:57:09.348 INFO 1433467 --- [pool-Check host task for agent18,3a60b1451fd548d2,edd2ec1a6a0e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:57:09.348 INFO 1433467 --- [pool-Check host task for agent28,3a60b1451fd548d2,6f05598202a7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:57:09.348 INFO 1433467 --- [pool-Check host task for agent28,3a60b1451fd548d2,6f05598202a7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:57:09.348 INFO 1433467 --- [pool-Check host task for agent28,3a60b1451fd548d2,6f05598202a7] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:57:09.349 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:57:09.358 WARN 1433467 --- [pool-Check host task for clockdiff26,44015733ab084554,a407c7335a7e] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.358 WARN 1433467 --- [pool-Check host task for clockdiff10,44015733ab084554,c737bc2ac126] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.358 ERROR 1433467 --- [pool-Check host task for clockdiff26,44015733ab084554,a407c7335a7e] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:57:09.358 WARN 1433467 --- [pool-Check host task for clockdiff14,44015733ab084554,4d8fa2d64d20] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.358 ERROR 1433467 --- [pool-Check host task for clockdiff10,44015733ab084554,c737bc2ac126] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:57:09.358 ERROR 1433467 --- [pool-Check host task for clockdiff14,44015733ab084554,4d8fa2d64d20] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:57:09.358 WARN 1433467 --- [pool-Check host task for clockdiff26,44015733ab084554,a407c7335a7e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.358 WARN 1433467 --- [pool-Check host task for clockdiff10,44015733ab084554,c737bc2ac126] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.358 WARN 1433467 --- [pool-Check host task for clockdiff14,44015733ab084554,4d8fa2d64d20] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.358 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.358 WARN 1433467 --- [pool-Check host task for clockdiff5,44015733ab084554,2b002a3470d0] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.358 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.358 ERROR 1433467 --- [pool-Check host task for clockdiff5,44015733ab084554,2b002a3470d0] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:57:09.358 WARN 1433467 --- [pool-Check host task for clockdiff5,44015733ab084554,2b002a3470d0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.358 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.358 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.358 WARN 1433467 --- [pool-Check host task for clockdiff11,44015733ab084554,29ccd0ca6832] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.358 ERROR 1433467 --- [pool-Check host task for clockdiff11,44015733ab084554,29ccd0ca6832] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:57:09.358 WARN 1433467 --- [pool-Check host task for clockdiff11,44015733ab084554,29ccd0ca6832] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.358 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:57:09.358 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:57:09.371 INFO 1433467 --- [pool-Check host task for agent30,3a60b1451fd548d2,d48d200057a2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:57:09.371 INFO 1433467 --- [pool-Check host task for agent18,3a60b1451fd548d2,edd2ec1a6a0e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:57:09.371 INFO 1433467 --- [pool-Check host task for agent18,3a60b1451fd548d2,edd2ec1a6a0e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:57:09.371 INFO 1433467 --- [pool-Check host task for agent30,3a60b1451fd548d2,d48d200057a2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:57:09.371 INFO 1433467 --- [pool-Check host task for agent18,3a60b1451fd548d2,edd2ec1a6a0e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:57:09.371 INFO 1433467 --- [pool-Check host task for agent30,3a60b1451fd548d2,d48d200057a2] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:57:09.371 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:57:09.372 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:57:09.372 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:57:09.372117946+08:00 2024-03-13 15:57:09.373 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:57:09.372+08:00 2024-03-13 15:57:09.375 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:57:09.372+08:00 and status AVAILABLE 2024-03-13 15:57:09.375 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:57:09.375 INFO 1433467 --- [pool-Check host task for agent20,3a60b1451fd548d2,4f6b5a3bdd24] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:57:09.376 INFO 1433467 --- [pool-Check host task for agent15,3a60b1451fd548d2,76d005bcd2e3] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:57:09.377 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:57:09.378 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:57:09.378 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:57:09.379 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:57:09.378802091+08:00 2024-03-13 15:57:09.379 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:57:09.379341040+08:00 2024-03-13 15:57:09.379 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:57:09.380 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:57:09.379667415+08:00 2024-03-13 15:57:09.380 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:57:09.379+08:00 2024-03-13 15:57:09.381 INFO 1433467 --- [pool-Check host task for agent20,3a60b1451fd548d2,4f6b5a3bdd24] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:57:09.381 INFO 1433467 --- [pool-Check host task for agent20,3a60b1451fd548d2,4f6b5a3bdd24] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:57:09.381 INFO 1433467 --- [pool-Check host task for agent20,3a60b1451fd548d2,4f6b5a3bdd24] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:57:09.381 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:57:09.379+08:00 2024-03-13 15:57:09.382 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:57:09.381448789+08:00 2024-03-13 15:57:09.382 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:57:09.379+08:00 and status AVAILABLE 2024-03-13 15:57:09.383 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:57:09.379+08:00 and status AVAILABLE 2024-03-13 15:57:09.384 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:57:09.385 INFO 1433467 --- [pool-Check host task for agent15,3a60b1451fd548d2,76d005bcd2e3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:57:09.385 INFO 1433467 --- [pool-Check host task for agent15,3a60b1451fd548d2,76d005bcd2e3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:57:09.385 INFO 1433467 --- [pool-Check host task for agent15,3a60b1451fd548d2,76d005bcd2e3] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:57:09.385 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:57:09.380+08:00 2024-03-13 15:57:09.386 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:57:09.382+08:00 2024-03-13 15:57:09.387 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:57:09.380+08:00 and status AVAILABLE 2024-03-13 15:57:09.388 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:57:09.382+08:00 and status AVAILABLE 2024-03-13 15:57:09.388 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:57:09.389 INFO 1433467 --- [pool-Check host task for agent2,3a60b1451fd548d2,cc63146357d8] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:57:09.392 INFO 1433467 --- [pool-Check host task for agent2,3a60b1451fd548d2,cc63146357d8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:57:09.392 INFO 1433467 --- [pool-Check host task for agent2,3a60b1451fd548d2,cc63146357d8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:57:09.392 INFO 1433467 --- [pool-Check host task for agent2,3a60b1451fd548d2,cc63146357d8] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:57:09.393 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:57:09.393 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:57:09.393 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:57:09.394 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:57:09.397 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:57:09.398 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:57:09.398 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:57:09.399 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:57:09.399 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:57:09.399 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:57:09.400 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:57:09.401 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:57:09.413 WARN 1433467 --- [pool-Check host task for clockdiff25,44015733ab084554,2e9900b7ed83] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.414 ERROR 1433467 --- [pool-Check host task for clockdiff25,44015733ab084554,2e9900b7ed83] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:57:09.414 WARN 1433467 --- [pool-Check host task for clockdiff25,44015733ab084554,2e9900b7ed83] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.430 WARN 1433467 --- [pool-Check host task for clockdiff19,44015733ab084554,c9cfdd191ed9] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.430 ERROR 1433467 --- [pool-Check host task for clockdiff19,44015733ab084554,c9cfdd191ed9] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:57:09.430 WARN 1433467 --- [pool-Check host task for clockdiff19,44015733ab084554,c9cfdd191ed9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:09.437 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:57:09.440 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:57:09.442 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:09.443 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:57:09.444 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:57:09.449 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:09.451 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:09.451 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:09.454 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:09.458 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:09.458 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:09.466 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:57:09.469 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:57:09.475 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:57:09.482 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:57:09.486 WARN 1433467 --- [pool-scanner-1609802826-1,c39e0b2f8aa04e72,5ac1d506c151] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:57:09.488 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:57:09.488 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:57:09.490 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:57:09.494 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:57:09.496 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:57:09.497 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:57:09.500 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:57:09.501 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:57:09.507 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:09.507 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:57:09.508 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:57:09.509 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.512 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.512 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.512 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:57:09.513 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:09.515 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:09.515 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:09.516 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:09.516 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:09.521 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:09.525 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:09.529 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:57:09.532 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:57:09.538 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:57:09.543 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:09.543 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.543 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:57:09.543 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.543 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.544 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:57:09.544 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:57:09.544 WARN 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:57:09.545 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:57:09.548 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.548 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.549 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:57:09.550 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:57:09.552 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:57:09.554 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:09.556 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:57:09.560 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:57:09.561 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:09.562 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:57:09.567 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:09.567 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:57:09.568 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:09.568 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:09.568 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:57:09.569 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:57:09.570 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:57:09.575 INFO 1433467 --- [pool-Check host task for host11,1d5c408fbda04de5,b4480cd9bb43] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:57:09.577 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:57:09.582 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:57:09.584 INFO 1433467 --- [pool-Check host task for host2,1d5c408fbda04de5,c15c1ac32349] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:57:09.584 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:09.587 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:09.588 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:09.588 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:57:09.595 INFO 1433467 --- [pool-Check host task for host38,1d5c408fbda04de5,3d2a4cca7d9e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:57:09.599 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:09.600 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.601 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.601 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.601 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:57:09.601 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:57:09.602 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:57:09.605 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:57:09.605 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:57:09.609 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:57:09.610 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:57:09.613 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:09.613 INFO 1433467 --- [pool-Check host task for host47,1d5c408fbda04de5,b6e9188d5036] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:57:09.623 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:57:09.627 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:57:09.630 INFO 1433467 --- [pool-Check host task for host14,1d5c408fbda04de5,54069e674fed] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:57:09.630 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:57:09.632 INFO 1433467 --- [pool-Check host task for host57,1d5c408fbda04de5,2b375a8cc093] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:57:09.657 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:57:09.663 INFO 1433467 --- [pool-Check host task for host18,1d5c408fbda04de5,5758c39843b1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:57:09.670 INFO 1433467 --- [alarm-detect-10,599c1522bec549b2,bd705f31cc67] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:57:09.673 INFO 1433467 --- [alarm-detect-10,599c1522bec549b2,bd705f31cc67] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:57:09.674 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:09.675 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.675 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.675 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.675 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:57:09.698 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:09.698 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.698 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.698 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.698 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:57:09.700 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:57:09.702 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.702 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.702 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:57:09.705 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:57:09.713 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:09.713 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.717 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:09.717 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.717 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:57:09.724 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:57:09.724 INFO 1433467 --- [ForkJoinPool-1386588-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316540069000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:57:09.725 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:09.725 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:57:09.739 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:09.739 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:57:09.746 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:09.746 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:57:09.754 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:09.754 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:57:09.761 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:09.761 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:57:09.768 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:09.775 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:09.775 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.776 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.776 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.776 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:57:09.780 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:57:09.782 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:57:09.784 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:57:09.785 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:57:09.798 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:57:09.833 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:57:09.855 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:09.856 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.857 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:09.857 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:09.858 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:57:09.875 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:57:09.875 INFO 1433467 --- [ForkJoinPool-1386588-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:57:09.918 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806209,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:10.198 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:10.199 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:10.398 INFO 1433467 --- [pool-async-queue-offer-interval60-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 60, queue size = 6, ruleSize = 6. 2024-03-13 15:57:10.407 INFO 1433467 --- [pool-async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 17, ruleSize = 17. 2024-03-13 15:57:10.927 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:57:10.927 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:57:10.932 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:57:10.932 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:57:10.936 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:57:10.936 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:57:10.940 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:57:10.940 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:57:11.238 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:57:11.239 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:57:11.240 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:57:11.261 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:57:11.262 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:57:11.263 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:57:11.284 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:57:11.285 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:57:11.301 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:57:11.302 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:57:11.303 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:57:11.303 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:57:11.303 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:57:11.303 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:57:11.303 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:57:11.303 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:57:11.310 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:57:11.311 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:57:11.318 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:57:11.318 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:57:11.318 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:57:11.318 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:57:11.318 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:57:11.318 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:57:11.318 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:57:11.319 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:57:11.321 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:57:11.321 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:57:11.321 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total 2024-03-13 15:57:11.321 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_route 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_server_process_request_time 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_hit 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_insert 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_update 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_response 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_sequence 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_partition 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_table 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_request 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_request 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_response 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_commit 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_delete 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_update 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_select 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_slow 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_others 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_sequence 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_large 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_prepare_send_request_time 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_success 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_select 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:57:11.326 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_delete 2024-03-13 15:57:11.327 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_commit 2024-03-13 15:57:11.327 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_others 2024-03-13 15:57:11.327 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_insert 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:57:11.329 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:57:11.332 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:57:11.335 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:57:11.338 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:57:11.341 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:57:11.344 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:57:11.347 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:57:11.350 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:57:11.353 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:57:11.919 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806232,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:12.370 INFO 1433467 --- [pool-metric-archive-second-3,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316560, nodeCount=1000, affectRows=1000 2024-03-13 15:57:12.374 INFO 1433467 --- [pool-metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316560, nodeCount=479, affectRows=479 2024-03-13 15:57:12.375 INFO 1433467 --- [pool-metric-archive-second-3,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316560, nodeCount=1000, validCount=1000 2024-03-13 15:57:12.378 INFO 1433467 --- [pool-metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316560, nodeCount=479, validCount=479 2024-03-13 15:57:12.383 INFO 1433467 --- [pool-metric-archive-second-2,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316560, nodeCount=540, affectRows=540 2024-03-13 15:57:12.387 INFO 1433467 --- [pool-metric-archive-second-2,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316560, nodeCount=539, validCount=539 2024-03-13 15:57:12.392 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316560, nodeCount=1000, affectRows=1000 2024-03-13 15:57:12.400 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316560, nodeCount=1000, validCount=1000 2024-03-13 15:57:12.416 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,cfefb258ad6441c7,5801f194af27] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.142, traceId=cfefb258ad6441c7, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:57:12.421 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,cfefb258ad6441c7,5801f194af27] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.142, traceId=cfefb258ad6441c7, duration=7 ms] 2024-03-13 15:57:13.200 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:13.201 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:13.226 INFO 1433467 --- [pool-ocp-schedules-16,916795e7f43f424a,ea3bb2264277] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:57:13.233 INFO 1433467 --- [pool-ocp-schedules-16,916795e7f43f424a,ea3bb2264277] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:57:13.233 INFO 1433467 --- [pool-ocp-schedules-16,916795e7f43f424a,ea3bb2264277] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:57:13.919 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806256,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:15.103 INFO 1433467 --- [pool-prometheus-second-schedule-1,8220c43d39ec430c,7103f7282c91] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=23, collectTaskCount=23 2024-03-13 15:57:15.919 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806279,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:16.109 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806282,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:57:16.181 INFO 1433467 --- [pool-ocp-schedules-17,a5d01affdaa8491c,5c590cc07ec4] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":12077,"minuteWriteQueue":0,"minuteReadCacheSize":12328,"secondWriteCacheSize":2485,"seriesIdKeySize":67930,"secondWriteQueue":0,"minuteWriteCacheSize":12404} 2024-03-13 15:57:16.181 INFO 1433467 --- [pool-ocp-schedules-17,a5d01affdaa8491c,5c590cc07ec4] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{go_memory_classes_heap_stacks_bytes=15, ob_sysstat_40012=19, ob_server_resource_memory_assigned_bytes=13, ob_sysstat_40011=19, ob_sysstat_40010=19, ob_unit_num=13, ob_database_disk_required_size=35, go_memstats_mspan_sys_bytes=15, go_memstats_stack_sys_bytes=15, ob_tenant_request_queue_total_size=18, jvm_gc_memory_allocated_bytes_total=3, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=18, ob_sysstat_40019=19, observer_fd_count=13, ob_sysstat_40018=19, processor_process_metrics_total=21, jvm_gc_max_data_size_bytes=3, go_memstats_frees_total=15, sql_audit_input_collect_data_jumped_total=8, host_available=64, mysql_output_write_sql_failed_count=17, ob_database_disk_data_size=34, ob_system_event_detail_total_timeouts=763, ob_tenant_memstore_freeze_cnt=14, ob_latch_sleep_total=312, ob_tenant_unit_min_memory=25, ob_sysstat_60087=19, ob_memstore_total_bytes=20, metric_cardinality_count=12, process_resident_memory_bytes=15, jvm_threads_states_threads=8, ob_cache_size_bytes=34, ob_tenant_unit_max_memory=25, input_collect_metrics_total=26, ob_sysstat_60083=19, ob_sysstat_50039=19, ob_sysstat_50038=19, node_disk_io_time_weighted_seconds_total=33, process_open_fds=15, ob_sysstat_50037=19, process_memory_rss_bytes=33, ob_server_resource_cpu_assigned_percent=13, ob_sysstat_50030=19, go_memory_classes_profiling_buckets_bytes=15, go_memstats_heap_inuse_bytes=15, ob_sysstat_40118=19, ob_sysstat_40117=19, ob_sysstat_40116=19, node_memory_memfree_bytes=26, ob_sysstat_40005=19, process_exists=43, ob_sysstat_40004=19, ob_sysstat_40003=19, ob_sysstat_40002=19, ob_sysstat_40001=19, go_gc_duration_seconds=20, go_memory_classes_metadata_mspan_inuse_bytes=15, ob_server_resource_disk_bytes=13, ob_sysstat_40000=19, odp_current_session=26, process_thread_count=33, ob_sysstat_50040=19, ob_sysstat_40009=19, ob_sysstat_40008=19, ob_disk_free_bytes=13, node_filesystem_avail_bytes=72, ob_sysstat_40007=19, ob_sysstat_40006=19, ob_backup_storage_capacity_bytes=650, ob_unit_config_max_iops=21, ob_plan_cache_memory_bytes=18, node_network_transmit_errs_total=28, go_gc_duration_seconds_count=15, monagent_oceanbase_plan_explain_size=4, ob_sysstat_50011=19, ob_sysstat_50010=19, node_disk_read_bytes_total=33, ob_compaction_is_suspended=19, mysql_output_write_sql_milliseconds_summary_count=17, go_memstats_lookups_total=15, mysql_output_write_sql_count=17, process_cpu_percent=33, ob_sysstat_60057=19, host_agent_process_status=18, ob_server_resource_cpu=13, ob_tenant_server_data_index_row_count=20, go_memstats_alloc_bytes=15, go_sched_gomaxprocs_threads=15, node_cpu_seconds_total_native=38, ob_sysstat_140002=19, ob_sysstat_140005=19, ob_sysstat_140006=19, ob_sysstat_140003=19, ob_tenant_resource_min_memory=18, ob_memstore_freeze_times=20, ob_sysstat_50022=19, jvm_gc_pause_seconds_count=8, ob_sysstat_50021=19, jvm_gc_overhead_percent=3, go_memory_classes_heap_unused_bytes=15, ob_cluster_status_check=13, jvm_memory_committed_bytes=10, node_net_bandwidth_bps=29, monagent_oceanbase_db_pool_idle=4, ob_tenant_server_data_index_data_size=20, go_gc_heap_allocs_by_size_bytes_total_count=15, go_memstats_heap_idle_bytes=15, go_gc_cycles_forced_gc_cycles_total=15, go_gc_pauses_seconds_total_bucket=26, ob_server_resource_memory_bytes=13, ob_tenant_assigned_mem_assigned=18, ob_tenant_memstore_minor_freeze_times=14, ob_tenant_server_required_size=22, ob_sysstat_60031=19, process_fd_count=33, ob_tenant_unit_max_cpu=25, ob_latch_wait_seconds_total=312, go_memory_classes_metadata_mcache_free_bytes=15, node_memory_buffers_bytes=26, ob_sysstat_50009=19, go_memstats_heap_released_bytes=15, node_network_receive_bytes_total=28, ob_sysstat_50005=19, ob_sysstat_50008=19, ob_sysstat_50001=19, ob_sysstat_50004=19, exposer_request_latencies_count=6, ob_sysstat_50000=19, ob_session_active_num=18, mysql_output_context_timeout_discard_metrics_count=9, go_memory_classes_metadata_mcache_inuse_bytes=15, node_disk_read_time_seconds_total=33, ob_sysstat_60021=19, ob_sysstat_60020=19, ob_sysstat_60024=19, ob_compaction_frozen_version=19, ob_sysstat_60023=19, metric_count=15, ob_sysstat_60022=19, obproxysys_connectable=1556, ob_tenant_assigned_cpu_assigned=18, mysql_output_write_sql_retry_count=17, ob_tenant_unit_min_cpu=25, ob_unit_config_max_cpu=21, http_request_milliseconds_summary_count=75, monagent_oceanbase_audit_timeinfo_cache_size=4, http_request_milliseconds_summary=78, ob_latch_immediate_miss_total=312, ob_latch_spin_get_total=312, ob_log_level_count=17, monagent_pipeline_report_metrics_total=16, exposer_transferred_bytes_total=6, ob_tenant_context_memory_count=841, node_disk_reads_completed_total=33, ob_sysstat_60019=19, ob_sysstat_60003=19, ob_sysstat_60002=19, ob_sysstat_60001=19, ob_sysstat_60000=19, ob_latch_miss_total=312, go_sched_latencies_seconds_count=15, ocp_meta_db_health=5, ob_disk_total_bytes=13, go_memstats_heap_objects=15, go_memstats_mcache_inuse_bytes=15, ob_sysstat_60005=19, ob_sysstat_60004=19, go_memory_classes_other_bytes=15, ob_memstore_active_bytes=20, go_memstats_alloc_bytes_total=15, ob_tenant_server_partition_replica_count=21, process_count=34, ob_system_event_detail_total_waits=763, odp_used_connections=25, backup_storage_capacity_used_percent=8, go_gc_heap_frees_objects_total=15, go_memory_classes_metadata_mspan_free_bytes=15, ob_disk_log_used_bytes=13, ob_compaction_is_error=19, go_gc_limiter_last_enabled_gc_cycle=15, monagent_pipeline_execute_total=24, node_filesystem_files=86, process_start_time_seconds=15, go_memstats_heap_alloc_bytes=15, ob_tenant_resource_max_cpu=18, ntp_service_available=26, go_memstats_mspan_inuse_bytes=15, mysql_output_write_sql_milliseconds_summary=20, go_gc_heap_frees_by_size_bytes_total_count=15, monagent_pipeline_execute_seconds_total=24, ob_sysstat_80001=19, go_memstats_sys_bytes=15, go_memory_classes_heap_free_bytes=15, system_load_average_1m=3, monagent_oceanbase_audit_stat_cache_size=4, ob_tenant_memory_used_bytes=22, obproxy_server_max_connections=7, ob_sysstat_80002=19, go_memstats_gc_cpu_fraction=15, ob_sysstat_80003=19, monagent_oceanbase_audit_cache_size=4, ob_tenant_context_memory_used_bytes=841, ob_sysstat_80007=19, node_disk_writes_completed_total=33, ob_sysstat_80008=19, ob_sysstat_80009=19, go_sched_goroutines_goroutines=15, go_memory_classes_metadata_other_bytes=15, go_gc_cycles_total_gc_cycles_total=15, system_cpu_usage=3, process_cpu_usage=3, node_disk_written_bytes_total=33, monagent_oceanbase_ash_cache_size=4, go_goroutines=15, ob_sysstat_80023=19, exposer_request_latencies_sum=6, ob_unit_config_min_cpu=21, process_virtual_memory_max_bytes=15, go_gc_heap_frees_bytes_total=15, ob_tenant500_memory_hold_bytes=13, exposer_scrapes_total=6, go_gc_heap_allocs_by_size_bytes_total_bucket=27, ob_sysstat_80025=19, ob_sysstat_80026=19, oceanbase_connectivity=15, node_memory_sreclaimable_bytes=26, go_sched_latencies_seconds_sum=15, tcp_retrans=25, exporter_export_metrics_bytes_total=26, node_filesystem_size_bytes=72, http_server_requests_seconds_count=203, ob_compaction_frozen_time=19, monagent_input_sql_duration_seconds_sum=55, log_stream_degraded_count=18, ob_plan_cache_hit_total=18, node_filesystem_readonly=72, ob_sysstat_80040=19, ob_sysstat_80041=19, io_await=52, go_gc_heap_allocs_objects_total=15, go_memstats_mallocs_total=15, ob_unit_config_max_memory_bytes=21, go_gc_stack_starting_size_bytes=15, io_util=52, ob_sysstat_200002=19, ob_sysstat_200001=19, ob_tenant_disk_v4_data_size=18, go_sched_latencies_seconds_bucket=26, ob_tenant_assigned_cpu_total=18, ob_tenant_server_available_disk_free=19, node_disk_write_time_seconds_total=33, cpu_count=25, output_write_metrics_total=9, ob_cluster_sync=15, go_gc_heap_objects_objects=15, odp_transaction_total=16, ob_system_event_detail_time_waited_us=763, process_max_fds=15, http_status_count=20, ob_sysstat_80057=19, ocp_alarm_detect_duration_ms_count=115, go_gc_heap_allocs_by_size_bytes_total_sum=15, go_memstats_mcache_sys_bytes=15, ob_tenant_disk_v4_log_size=18, go_cgo_go_to_c_calls_calls_total=15, ocp_alarm_detect_duration_ms_sum=115, jvm_threads_live_threads=3, node_memory_cached_bytes=26, ob_table_num=16, ob_sysstat_10000=19, ob_sysstat_10006=19, ob_sysstat_10005=19, ob_sysstat_10002=19, ob_sysstat_10001=19, go_memstats_last_gc_time_seconds=15, ob_sysstat_10004=19, ob_sysstat_10003=19, monitor_exporter_avaliable=62, log_tailer_count=15, ob_server_num=25, ob_session_all_num=18, ob_system_event_time_waited=24, ob_tenant_resource_min_cpu=18, monagent_plugin_execute_seconds_total=39, go_gc_cycles_automatic_gc_cycles_total=15, go_gc_heap_allocs_bytes_total=15, go_memstats_gc_sys_bytes=15, ob_latch_get_total=312, go_info=16, log_tailer_processed_file_count=21, node_load15=24, ob_disk_log_assigned_bytes=13, go_memory_classes_heap_released_bytes=15, jvm_memory_max_bytes=10, ob_unit_config_min_iops=21, ob_sysstat_30080=19, ob_sysstat_30082=19, ob_sysstat_30081=19, ob_sysstat_30084=19, ob_sysstat_30083=19, ob_sysstat_30086=19, ob_sysstat_30085=19, ob_tenant_log_disk_total_bytes=18, ob_memstore_freeze_trigger_bytes=20, ob_waitevent_wait_seconds_total=18, ob_server_stopped_duration_seconds=15, go_memstats_next_gc_bytes=15, node_filesystem_files_free=86, ob_memtable_max_snapshot_duration_seconds=18, monagent_oceanbase_audit_unique_key_cache_size=4, process_virtual_memory_bytes=15, ob_sysstat_20002=19, net_connectivity=15, ob_sysstat_20001=19, monagent_input_sql_duration_seconds_bucket=65, ob_latch_immediate_get_total=312, node_cpu_seconds_total=32, ob_sysstat_20000=19, ob_index_error_num=13, ob_tenant_server_available_disk_total=19, ob_tenant_memory_hold_bytes=22, ob_sysstat_30066=19, go_memstats_buck_hash_sys_bytes=15, mysql_output_write_sql_milliseconds_summary_sum=17, ob_tenant_disk_total_size=18, odp_sql_request_total=53, mysql_output_write_task_size=17, exporter_export_metrics_total=26, ob_disk_log_total_bytes=13, system_cpu_count=3, ob_sysstat_130004=19, process_uptime_seconds=33, node_ntp_offset_seconds=25, go_memstats_heap_sys_bytes=15, ob_sysstat_140012=19, ob_sysstat_140013=19, jvm_gc_pause_seconds_max=8, ob_tenant_request_queue_large_size=18, ob_sysstat_130000=19, go_memstats_other_sys_bytes=15, ob_sysstat_130001=19, jvm_memory_used_bytes=10, ob_sysstat_130002=19, go_threads=15, http_request_milliseconds_summary_sum=75, http_server_requests_seconds_sum=203, ob_tenant_assigned_mem_total=18, node_memory_memtotal_bytes=26, node_network_transmit_drop_total=28, go_gc_pauses_seconds_total_count=15, exposer_request_latencies=9, ob_waitevent_wait_total=18, go_gc_heap_goal_bytes=15, ob_tenant_log_disk_used_bytes=18, jvm_gc_memory_promoted_bytes_total=3, odp_sql_cost_total=51, ob_sysstat_30011=19, ob_sysstat_30010=19, ob_sysstat_30013=19, ob_sysstat_30012=19, ob_tenant500_memory_used_bytes=13, node_network_transmit_bytes_total=28, node_network_receive_drop_total=28, go_gc_heap_tiny_allocs_objects_total=15, go_gc_duration_seconds_sum=15, ob_server_resource_memory_assigned_percent=13, jvm_gc_pause_seconds_sum=8, go_memory_classes_heap_objects_bytes=15, node_network_receive_errs_total=28, observer_ulimit_max_fd_count=13, ob_plan_cache_access_total=18, ob_tenant_server_row_count=22, monagent_oceanbase_plan_cache_size=4, ob_tenant_server_data_size=22, monagent_oceanbase_audit_sample_cache_size=4, go_gc_heap_frees_by_size_bytes_total_sum=15, ob_sysstat_40030=19, node_load1=24, process_cpu_seconds_total=15, ob_sysstat_120001=19, go_memory_classes_total_bytes=15, ob_sysstat_120002=19, ob_sysstat_120000=19, ob_tenant_resource_max_memory=18, ob_sysstat_120009=19, go_memstats_stack_inuse_bytes=15, ob_sysstat_120008=19, node_load5=24, ob_sysstat_120005=19, go_memory_classes_os_stacks_bytes=15, ob_sysstat_120006=19, jvm_threads_peak_threads=3, monagent_input_sql_duration_seconds_count=55, ob_tenant_context_memory_hold_bytes=841, obproxy_memory_limit_bytes=7, ob_compaction_last_version=19, ob_sysstat_30000=19, ob_server_resource_cpu_assigned=13, ob_sysstat_30002=19, ob_sysstat_30001=19, monagent_oceanbase_plan_timeinfo_cache_size=4, monagent_plugin_execute_total=39, ob_sysstat_30006=19, ob_sysstat_30005=19, ob_sysstat_30008=19, ob_sysstat_30007=19, ob_sysstat_30009=19, go_gc_heap_frees_by_size_bytes_total_bucket=27, obproxy_client_max_connections=7, ob_compaction_current_timestamp=19, ob_clog_max_sync_delay_seconds=20, go_gc_pauses_seconds_total_sum=15, ob_compaction_start_time=19} 2024-03-13 15:57:16.201 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:16.202 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:16.747 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:57:16.747096+08:00, lastProcessTime=2024-03-13T15:57:06.748413+08:00 2024-03-13 15:57:16.858 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:57:16.747096+08:00 2024-03-13 15:57:17.011 INFO 1433467 --- [pool-ocp-schedules-12,4e1626fadcd04230,fee055e7fc2c] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:57:17.920 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806303,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:19.202 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:19.202 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:19.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806318,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:57:19.281 INFO 1433467 --- [alarm-detect-20,d912ffb9fbea43f0,943d87555fe5] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:57:19.283 INFO 1433467 --- [alarm-detect-26,b9a26a6b3538417b,d233ad4c5eae] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:57:19.283 INFO 1433467 --- [alarm-detect-2,532be73c977d4fe7,ee9b579c1be4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:57:19.285 INFO 1433467 --- [alarm-detect-20,d912ffb9fbea43f0,943d87555fe5] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:57:19.286 INFO 1433467 --- [alarm-detect-26,b9a26a6b3538417b,d233ad4c5eae] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:57:19.287 INFO 1433467 --- [alarm-detect-2,532be73c977d4fe7,ee9b579c1be4] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:57:19.324 WARN 1433467 --- [pool-scanner-1609804389-1,c62372b2626f4a87,294e87a4be41] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:57:19.424 INFO 1433467 --- [alarm-detect-17,354ce64577634cfc,22dbaf0483bd] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316605 ~ 1710316629 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316620=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:57:19.424 INFO 1433467 --- [alarm-detect-17,354ce64577634cfc,22dbaf0483bd] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316605 ~ 1710316629 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316620=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:57:19.424 INFO 1433467 --- [alarm-detect-17,354ce64577634cfc,22dbaf0483bd] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316605 ~ 1710316629 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316620=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:57:19.424 INFO 1433467 --- [alarm-detect-17,354ce64577634cfc,22dbaf0483bd] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316605 ~ 1710316629 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316620=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:57:19.425 INFO 1433467 --- [alarm-detect-17,354ce64577634cfc,22dbaf0483bd] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316605 ~ 1710316629 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316620=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:57:19.920 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806327,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:20.482 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:21.0, nextRun=2024-03-13 15:57:20.0, currentDate=Wed Mar 13 15:57:20 CST 2024 2024-03-13 15:57:20.483 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999996, name=Collect all cluster compaction info, templateName=CollectClusterCompactionSchedule, scheduleRule=20 * * * * *, lastRunTime=2024-03-13 15:56:21.0, nextRunTime=2024-03-13 15:57:20.0} 2024-03-13 15:57:20.485 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all cluster compaction info, nodeCount=1, taskCount=1 2024-03-13 15:57:20.485 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:57:20.485 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all cluster compaction info, taskDefinitionId=999996, state=RUNNING, operation=EXECUTE} 2024-03-13 15:57:21.501 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016765, operation:EXECUTE, state: RUNNING 2024-03-13 15:57:21.525 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,0aab68799e8a] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016765, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014413, task_operation=execute, latest_execution_start_time=2024-03-13T15:57:21.499+08:00, sub_task_instance_name=Collect compaction for all clusters, sub_task_instance_id=10016765}, listMap={}}, executor=10.10.10.148 2024-03-13 15:57:21.526 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,0aab68799e8a] c.o.o.s.t.b.c.AbstractAllClusterTask : Collect compaction for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:57:21.527 INFO 1433467 --- [ForkJoinPool-1386593-worker-1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:57:21.527 INFO 1433467 --- [ForkJoinPool-1386593-worker-1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:57:21.532 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,0aab68799e8a] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016765, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:57:21.920 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806350,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:22.112 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,b54ba3660b774aec,77cd00aa11af] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.144, traceId=b54ba3660b774aec, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:57:22.117 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,b54ba3660b774aec,77cd00aa11af] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.144, traceId=b54ba3660b774aec, duration=6 ms] 2024-03-13 15:57:22.203 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:22.204 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:22.266 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014413, status:SUCCESS 2024-03-13 15:57:22.267 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014413, status:SUCCESS 2024-03-13 15:57:23.921 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806372,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:25.204 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:25.205 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:25.922 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806395,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:26.110 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806398,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:57:26.545 INFO 1433467 --- [pool-Check obproxy2,2c0a3b84e21041b8,8a32c1dbc294] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:57:26.545 INFO 1433467 --- [pool-Check obproxy13,2c0a3b84e21041b8,15c3220775e3] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:57:26.547 INFO 1433467 --- [pool-Check obproxy3,2c0a3b84e21041b8,4b99dc2f7ac2] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:57:26.549 INFO 1433467 --- [pool-Check obproxy6,2c0a3b84e21041b8,1d991f0b362f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:57:26.748 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:57:26.747482+08:00, lastProcessTime=2024-03-13T15:57:16.747096+08:00 2024-03-13 15:57:26.867 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:57:26.747482+08:00 2024-03-13 15:57:27.018 INFO 1433467 --- [pool-ocp-schedules-19,3f67c28b48834823,1161fa30c0e0] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:57:27.922 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806419,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:28.205 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:28.207 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:28.792 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:28.792 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:28.792 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:28.793 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:57:28.797 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:28.797 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:28.798 INFO 1433467 --- [sql-rollup-daemon,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:57:28.799 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule start 2024-03-13 15:57:28.817 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=1, name=obcluster1, obClusterId=1, status=RUNNING} 2024-03-13 15:57:28.828 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : got cluster:ObClusterEntity{id=2, name=obcluster, obClusterId=1, status=RUNNING} 2024-03-13 15:57:28.833 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster1:1-3) 2024-03-13 15:57:28.833 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster:1-1) 2024-03-13 15:57:28.833 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster1:1-1) 2024-03-13 15:57:28.834 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Submit concurrent group: (obcluster1:1-2) 2024-03-13 15:57:28.838 INFO 1433467 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:28.838 INFO 1433467 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:28.839 INFO 1433467 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:28.839 INFO 1433467 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=3, level=0->1, c=0) - not enough cached interval (source=2024-03-13T15:55:00, target=2024-03-13T15:53:00, granule=120000ms, delay=120000ms) 2024-03-13 15:57:28.839 INFO 1433467 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:57:28.839 INFO 1433467 --- [sql-rollup-worker-3,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=3, level=1->2, c=0) - not enough cached interval (source=2024-03-13T15:53:00, target=2024-03-13T15:46:00, granule=600000ms, delay=420000ms) 2024-03-13 15:57:28.841 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:28.841 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:28.841 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:28.841 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'version_comment' 2024-03-13 15:57:28.841 INFO 1433467 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:28.841 INFO 1433467 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:28.842 INFO 1433467 --- [sql-rollup-worker-0,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2024-03-13 15:57:28.842 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:28.842 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:28.842 INFO 1433467 --- [sql-rollup-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select now() 2024-03-13 15:57:28.879 INFO 1433467 --- [sql-rollup-worker-0,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=obcluster1:1, server=2, level=0->1, c=0) - source=2024-03-13T15:55:02, start=2024-03-13T15:52:32, end=2024-03-13T15:55:02, updates=124, granule=120000ms, delay=150000ms 2024-03-13 15:57:28.893 INFO 1433467 --- [sql-rollup-worker-0,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=2, level=0->1, c=1) - not enough cached interval (source=2024-03-13T15:55:02, target=2024-03-13T15:55:02, granule=120000ms, delay=0ms) 2024-03-13 15:57:28.904 INFO 1433467 --- [sql-rollup-worker-2,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=obcluster1:1, server=1, level=0->1, c=0) - source=2024-03-13T15:55:02, start=2024-03-13T15:52:32, end=2024-03-13T15:55:02, updates=256, granule=120000ms, delay=150000ms 2024-03-13 15:57:28.908 INFO 1433467 --- [sql-rollup-worker-2,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=1, level=0->1, c=1) - not enough cached interval (source=2024-03-13T15:55:02, target=2024-03-13T15:55:02, granule=120000ms, delay=0ms) 2024-03-13 15:57:28.908 INFO 1433467 --- [sql-rollup-worker-2,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=1, level=1->2, c=0) - not enough cached interval (source=2024-03-13T15:55:02, target=2024-03-13T15:43:02, granule=600000ms, delay=720000ms) 2024-03-13 15:57:28.940 INFO 1433467 --- [sql-rollup-worker-0,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Rollup server-side (cluster=obcluster1:1, server=2, level=1->2, c=0) - source=2024-03-13T15:55:02, start=2024-03-13T15:40:32, end=2024-03-13T15:50:32, updates=128, granule=600000ms, delay=870000ms 2024-03-13 15:57:28.943 INFO 1433467 --- [sql-rollup-worker-0,,] o.o.p.s.i.c.SqlAuditRawStatRollupService : Try rollup subsequent (cluster=obcluster1:1, server=2, level=1->2, c=1) - not enough cached interval (source=2024-03-13T15:55:02, target=2024-03-13T15:50:32, granule=600000ms, delay=270000ms) 2024-03-13 15:57:28.943 INFO 1433467 --- [sql-rollup-daemon,,] s.i.c.SqlAuditRawStatRollupDaemonService : Rollup schedule end, elapsed=144ms, period=120000ms 2024-03-13 15:57:29.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806435,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:57:29.321 WARN 1433467 --- [pool-scanner-1609805020-1,43e5200f003942df,39be42fc910c] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:57:29.413 INFO 1433467 --- [alarm-detect-16,33f4a16dbecc4a44,4afc374aab00] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:57:29.416 INFO 1433467 --- [alarm-detect-16,33f4a16dbecc4a44,4afc374aab00] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:57:29.449 INFO 1433467 --- [alarm-detect-15,4f9316daef8145c5,23ce7515679f] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316620 ~ 1710316639 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316635=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:57:29.449 INFO 1433467 --- [alarm-detect-15,4f9316daef8145c5,23ce7515679f] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316620 ~ 1710316639 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316635=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:57:29.449 INFO 1433467 --- [alarm-detect-15,4f9316daef8145c5,23ce7515679f] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316620 ~ 1710316639 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316635=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:57:29.449 INFO 1433467 --- [alarm-detect-15,4f9316daef8145c5,23ce7515679f] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316620 ~ 1710316639 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316635=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:57:29.449 INFO 1433467 --- [alarm-detect-15,4f9316daef8145c5,23ce7515679f] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316620 ~ 1710316639 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316635=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:57:29.449 INFO 1433467 --- [alarm-detect-15,4f9316daef8145c5,23ce7515679f] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316620 ~ 1710316639 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316635=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:57:29.449 INFO 1433467 --- [alarm-detect-15,4f9316daef8145c5,23ce7515679f] c.o.o.a.core.detect.AlarmRuleDetector : detectByMetric done, alarmType=host_ntp_service_not_exist, needSendingCount=1, detectAlarmsCount=1, matchedTargets=[alarm_template_id=3:host=10.10.10.148-2, alarm_template_id=3:host=10.10.10.147-2, alarm_template_id=3:host=10.10.10.146-2, alarm_template_id=3:host=10.10.10.145-2, alarm_template_id=3:host=10.10.10.144-2, alarm_template_id=3:host=10.10.10.143-2] 2024-03-13 15:57:29.923 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806443,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:30.831 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:31.0, nextRun=2024-03-13 15:57:30.0, currentDate=Wed Mar 13 15:57:30 CST 2024 2024-03-13 15:57:30.831 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999990, name=Sync tenant information, templateName=TenantSyncSchedule, scheduleRule=30 0/1 * * * ?, lastRunTime=2024-03-13 15:56:31.0, nextRunTime=2024-03-13 15:57:30.0} 2024-03-13 15:57:30.839 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Sync tenant information, nodeCount=1, taskCount=2 2024-03-13 15:57:30.839 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:57:30.839 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:57:30.839 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Sync tenant information, taskDefinitionId=999990, state=RUNNING, operation=EXECUTE} 2024-03-13 15:57:30.846 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:57:01.0, nextRun=2024-03-13 15:57:30.0, currentDate=Wed Mar 13 15:57:30 CST 2024 2024-03-13 15:57:30.846 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:57:01.0, nextRunTime=2024-03-13 15:57:30.0} 2024-03-13 15:57:30.848 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:57:30.848 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:57:30.848 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:57:31.207 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:31.208 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:31.614 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016766, operation:EXECUTE, state: RUNNING 2024-03-13 15:57:31.628 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016767, operation:EXECUTE, state: RUNNING 2024-03-13 15:57:31.638 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016766, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014414, task_operation=execute, latest_execution_start_time=2024-03-13T15:57:31.610+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016766}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:57:31.641 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016768, operation:EXECUTE, state: RUNNING 2024-03-13 15:57:31.649 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016767, context=Context{parallelIdx=1, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014414, task_operation=execute, latest_execution_start_time=2024-03-13T15:57:31.625+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016767}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:57:31.662 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,c4c3b81c5b97] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016768, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014415, task_operation=execute, latest_execution_start_time=2024-03-13T15:57:31.638+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016768}, listMap={}}, executor=10.10.10.148 2024-03-13 15:57:31.671 INFO 1433467 --- [pool-Check obproxy11,f47a761474a84dce,f7169019a8d3] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:57:31.675 INFO 1433467 --- [pool-Check obproxy8,f47a761474a84dce,cb264d775a8e] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:57:31.676 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,2c5969b2cd21] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:57:31.678 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,0be8d96afed3] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:57:31.682 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.691 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:57:31.691 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.691 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.692 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:57:31.696 INFO 1433467 --- [pool-Check obproxy11,f47a761474a84dce,f7169019a8d3] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:57:31.699 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.699 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.699 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:57:31.702 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.703 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:57:31.703 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.703 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.703 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:57:31.704 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,0be8d96afed3] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:57:31.705 INFO 1433467 --- [pool-Check obproxy8,f47a761474a84dce,cb264d775a8e] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:57:31.706 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.706 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.706 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:57:31.712 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.712 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.712 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:57:31.713 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,2c5969b2cd21] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:57:31.713 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.713 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.714 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:57:31.720 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.720 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.720 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:57:31.722 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,c4c3b81c5b97] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016768, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:57:31.724 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.724 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.724 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:57:31.733 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.733 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.733 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:57:31.739 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.739 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.739 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:57:31.743 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.743 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.747 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.747 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.748 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:57:31.757 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.758 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.758 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.758 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.759 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:57:31.767 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.767 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.767 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.767 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.768 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.768 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.768 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.768 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.768 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:57:31.768 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:57:31.783 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.783 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.783 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.783 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.783 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.783 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.784 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:57:31.784 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.784 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.785 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:57:31.794 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.794 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.795 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.795 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.795 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.795 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:57:31.796 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.796 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.796 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.796 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:57:31.804 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.805 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.805 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.805 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.806 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:57:31.806 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=2 2024-03-13 15:57:31.812 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.812 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.812 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.812 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.813 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:57:31.815 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:57:31.823 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.824 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.824 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.824 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.824 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.825 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.825 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:57:31.826 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.826 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.826 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:57:31.835 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=4, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:57:31.839 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=4, userWhitelist=% 2024-03-13 15:57:31.840 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:31.840 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.841 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:57:31.841 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.841 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.842 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.842 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.842 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:57:31.844 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2024-03-13 15:57:31.847 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:57:31.848 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=3, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:57:31.850 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=3, userWhitelist=% 2024-03-13 15:57:31.850 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=2 2024-03-13 15:57:31.850 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:57:31.855 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,e67755175414] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016767, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:57:31.869 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:31.870 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.871 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.871 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.872 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:57:31.886 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:57:31.887 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2024-03-13 15:57:31.887 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:31.887 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:31.888 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:57:31.894 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:57:31.895 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2024-03-13 15:57:31.895 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2024-03-13 15:57:31.896 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:57:31.901 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,1818db60bf48] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016766, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:57:31.923 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806466,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:32.449 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014414, status:SUCCESS 2024-03-13 15:57:32.450 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014414, status:SUCCESS 2024-03-13 15:57:32.460 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014415, status:SUCCESS 2024-03-13 15:57:32.460 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014415, status:SUCCESS 2024-03-13 15:57:33.236 INFO 1433467 --- [pool-ocp-schedules-11,cc9a02f675314e0a,2af5c7a0ab00] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:57:33.245 INFO 1433467 --- [pool-ocp-schedules-11,cc9a02f675314e0a,2af5c7a0ab00] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:57:33.245 INFO 1433467 --- [pool-ocp-schedules-11,cc9a02f675314e0a,2af5c7a0ab00] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:57:33.923 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806489,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:34.208 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:34.209 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:34.539 INFO 1433467 --- [pool-ocp-schedules-8,5d0ca596497d4c33,e8e77643edb6] c.o.o.core.cache.helper.HostQueryCache : Init host cache. 2024-03-13 15:57:34.546 INFO 1433467 --- [pool-ocp-schedules-8,5d0ca596497d4c33,e8e77643edb6] c.o.o.core.cache.helper.HostQueryCache : Finished init host cache. 2024-03-13 15:57:35.520 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,fa2925ead05a42ae,f5715727258a] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.148, traceId=fa2925ead05a42ae, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:57:35.527 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,fa2925ead05a42ae,f5715727258a] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=fa2925ead05a42ae, duration=8 ms] 2024-03-13 15:57:35.924 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806512,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:36.110 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806515,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:57:36.774 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:57:36.774316+08:00, lastProcessTime=2024-03-13T15:57:26.747482+08:00 2024-03-13 15:57:36.892 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:57:36.774316+08:00 2024-03-13 15:57:37.023 INFO 1433467 --- [pool-ocp-schedules-8,07763e47fee64420,45950415dcb4] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:57:37.209 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:37.212 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:37.924 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806536,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:39.254 INFO 1433467 --- [pool-ocp-schedules-18,5d9edfa60bc34f92,06e2d98a2095] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:57:39.256 INFO 1433467 --- [pool-ocp-schedules-4,1276d6fb03854659,a9de552ff18b] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:57:39.256 INFO 1433467 --- [pool-ocp-schedules-11,e7a2d89c9a584914,35af32363c36] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:57:39.256 INFO 1433467 --- [pool-ocp-schedules-18,5d9edfa60bc34f92,06e2d98a2095] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:57:39.265 INFO 1433467 --- [pool-ocp-schedules-11,e7a2d89c9a584914,35af32363c36] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:57:39.266 INFO 1433467 --- [pool-ocp-schedules-4,1276d6fb03854659,a9de552ff18b] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:57:39.274 INFO 1433467 --- [pool-ocp-schedules-17,63cbc4d112844c6a,d6ded380cbf1] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:57:39.278 INFO 1433467 --- [pool-ocp-schedules-15,27c263435776452d,6dc8cf2c2198] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:57:39.282 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806551,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:57:39.296 INFO 1433467 --- [alarm-detect-9,3f1a61969d33445f,a9646a17aad6] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:57:39.296 INFO 1433467 --- [alarm-detect-20,1f969702e0824f96,eceb610a7f27] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:57:39.297 INFO 1433467 --- [alarm-detect-11,f4175829bd0e4b28,bf55dedf54b2] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:57:39.302 INFO 1433467 --- [alarm-detect-9,3f1a61969d33445f,a9646a17aad6] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:57:39.303 INFO 1433467 --- [alarm-detect-20,1f969702e0824f96,eceb610a7f27] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:57:39.303 INFO 1433467 --- [alarm-detect-11,f4175829bd0e4b28,bf55dedf54b2] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:57:39.317 INFO 1433467 --- [pool-ocp-schedules-8,898edbfdd3684000,e287afcd80ff] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:57:39.317 INFO 1433467 --- [pool-Check host task for agent16,898edbfdd3684000,0efd199182da] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:57:39.317 INFO 1433467 --- [pool-Check host task for agent11,898edbfdd3684000,a43d1d8c3517] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:57:39.317 INFO 1433467 --- [pool-Check host task for agent17,898edbfdd3684000,103929c2d82e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:57:39.317 INFO 1433467 --- [pool-Check host task for agent6,898edbfdd3684000,ab578e3d1669] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:57:39.317 INFO 1433467 --- [pool-Check host task for agent24,898edbfdd3684000,fdd4c0fc5248] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:57:39.317 INFO 1433467 --- [pool-Check host task for agent22,898edbfdd3684000,5b77c520a8dd] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:57:39.317 INFO 1433467 --- [pool-Check host task for agent25,898edbfdd3684000,b9ba07d063ce] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:57:39.321 INFO 1433467 --- [pool-Check host task for agent17,898edbfdd3684000,103929c2d82e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:57:39.322 INFO 1433467 --- [pool-Check host task for agent6,898edbfdd3684000,ab578e3d1669] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:57:39.322 INFO 1433467 --- [pool-Check host task for agent24,898edbfdd3684000,fdd4c0fc5248] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:57:39.324 INFO 1433467 --- [pool-Check host task for agent25,898edbfdd3684000,b9ba07d063ce] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:57:39.326 INFO 1433467 --- [pool-Check host task for agent6,898edbfdd3684000,ab578e3d1669] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:57:39.327 INFO 1433467 --- [pool-ocp-schedules-9,70e3aa2894294a65,115a504611ee] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:57:39.327 INFO 1433467 --- [pool-Check host task for agent24,898edbfdd3684000,fdd4c0fc5248] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:57:39.327 INFO 1433467 --- [pool-Check host task for agent17,898edbfdd3684000,103929c2d82e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:57:39.327 INFO 1433467 --- [pool-Check host task for agent22,898edbfdd3684000,5b77c520a8dd] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:57:39.328 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:57:39.329 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:57:39.329 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:57:39.329 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:57:39.329 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:57:39.329 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:57:39.329 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:57:39.329 INFO 1433467 --- [pool-Check host task for agent16,898edbfdd3684000,0efd199182da] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:57:39.332 INFO 1433467 --- [pool-Check host task for agent25,898edbfdd3684000,b9ba07d063ce] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:57:39.334 INFO 1433467 --- [pool-Check host task for agent11,898edbfdd3684000,a43d1d8c3517] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:57:39.335 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:57:39.336 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:57:39.336 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:57:39.336 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:57:39.337 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:57:39.337 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:57:39.338 INFO 1433467 --- [pool-Check host task for agent11,898edbfdd3684000,a43d1d8c3517] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:57:39.339 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:57:39.339 INFO 1433467 --- [ForkJoinPool-1386596-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.339 INFO 1433467 --- [ForkJoinPool-1386596-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.339 INFO 1433467 --- [ForkJoinPool-1386596-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.340 INFO 1433467 --- [ForkJoinPool-1386596-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:57:39.340 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:57:39.341 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:57:39.342 INFO 1433467 --- [pool-Check host task for agent22,898edbfdd3684000,5b77c520a8dd] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:57:39.332 INFO 1433467 --- [pool-Check host task for agent16,898edbfdd3684000,0efd199182da] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:57:39.343 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:57:39.343 INFO 1433467 --- [ForkJoinPool-1386596-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:57:39.346 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:57:39.347 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:57:39.354 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:57:39.356 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:57:39.393 INFO 1433467 --- [ForkJoinPool-1386596-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.393 INFO 1433467 --- [ForkJoinPool-1386596-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.393 INFO 1433467 --- [ForkJoinPool-1386596-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.394 INFO 1433467 --- [ForkJoinPool-1386596-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:57:39.394 INFO 1433467 --- [ForkJoinPool-1386596-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:57:39.401 INFO 1433467 --- [pool-ocp-schedules-10,9a05e8bf9c114c78,afb718ddc5c6] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:57:39.401 INFO 1433467 --- [pool-Check host task for clockdiff13,9a05e8bf9c114c78,9595359e5c26] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:57:39.401 INFO 1433467 --- [pool-Check host task for clockdiff22,9a05e8bf9c114c78,9336a8c9b851] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:57:39.401 INFO 1433467 --- [pool-Check host task for clockdiff6,9a05e8bf9c114c78,d9a5caea90b5] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:57:39.401 INFO 1433467 --- [pool-Check host task for clockdiff29,9a05e8bf9c114c78,e827ca571f2a] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:57:39.401 INFO 1433467 --- [pool-Check host task for clockdiff24,9a05e8bf9c114c78,0b9c83a7e239] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:57:39.401 INFO 1433467 --- [pool-Check host task for clockdiff8,9a05e8bf9c114c78,a5f644a3af26] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:57:39.401 INFO 1433467 --- [pool-Check host task for clockdiff2,9a05e8bf9c114c78,abd12b5eb054] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:57:39.414 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.414 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.414 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.415 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.415 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:57:39.415 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.415 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.415 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:57:39.422 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:57:39.423 INFO 1433467 --- [pool-Check host task for agent22,898edbfdd3684000,5b77c520a8dd] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:57:39.423 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:57:39.424 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:57:39.424 INFO 1433467 --- [pool-Check host task for agent25,898edbfdd3684000,b9ba07d063ce] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:57:39.424 INFO 1433467 --- [pool-Check host task for agent6,898edbfdd3684000,ab578e3d1669] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:57:39.425 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:57:39.424796653+08:00 2024-03-13 15:57:39.425 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:57:39.425+08:00 2024-03-13 15:57:39.425 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:57:39.425 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:57:39.425647103+08:00 2024-03-13 15:57:39.425 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:57:39.425452499+08:00 2024-03-13 15:57:39.426 INFO 1433467 --- [pool-Check host task for agent16,898edbfdd3684000,0efd199182da] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:57:39.426 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:57:39.426+08:00 2024-03-13 15:57:39.426 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:57:39.426+08:00 2024-03-13 15:57:39.452 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:57:39.454 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:57:39.454041594+08:00 2024-03-13 15:57:39.454 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:57:39.454+08:00 2024-03-13 15:57:39.456 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:57:39.454+08:00 and status AVAILABLE 2024-03-13 15:57:39.459 INFO 1433467 --- [pool-Check host task for agent25,898edbfdd3684000,b9ba07d063ce] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:57:39.459 INFO 1433467 --- [pool-Check host task for agent6,898edbfdd3684000,ab578e3d1669] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:57:39.459 INFO 1433467 --- [pool-Check host task for agent25,898edbfdd3684000,b9ba07d063ce] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:57:39.459 INFO 1433467 --- [pool-Check host task for agent6,898edbfdd3684000,ab578e3d1669] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:57:39.459 INFO 1433467 --- [pool-Check host task for agent25,898edbfdd3684000,b9ba07d063ce] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:57:39.459 INFO 1433467 --- [pool-Check host task for agent6,898edbfdd3684000,ab578e3d1669] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:57:39.459 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:57:39.426+08:00 and status AVAILABLE 2024-03-13 15:57:39.464 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:57:39.465 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:57:39.465 INFO 1433467 --- [pool-Check host task for agent22,898edbfdd3684000,5b77c520a8dd] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:57:39.465 INFO 1433467 --- [pool-Check host task for agent22,898edbfdd3684000,5b77c520a8dd] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:57:39.465 INFO 1433467 --- [pool-Check host task for agent22,898edbfdd3684000,5b77c520a8dd] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:57:39.466 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:57:39.465842901+08:00 2024-03-13 15:57:39.466 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:57:39.466+08:00 2024-03-13 15:57:39.467 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:57:39.426+08:00 and status AVAILABLE 2024-03-13 15:57:39.467 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:57:39.466+08:00 and status AVAILABLE 2024-03-13 15:57:39.468 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:57:39.468 INFO 1433467 --- [pool-Check host task for agent16,898edbfdd3684000,0efd199182da] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:57:39.468 INFO 1433467 --- [pool-Check host task for agent16,898edbfdd3684000,0efd199182da] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:57:39.468 INFO 1433467 --- [pool-Check host task for agent16,898edbfdd3684000,0efd199182da] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:57:39.469 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:57:39.425+08:00 and status AVAILABLE 2024-03-13 15:57:39.470 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:57:39.469532481+08:00 2024-03-13 15:57:39.470 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:57:39.470+08:00 2024-03-13 15:57:39.472 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:57:39.473 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:57:39.473 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:57:39.474 INFO 1433467 --- [pool-Check host task for agent11,898edbfdd3684000,a43d1d8c3517] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:57:39.477 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:57:39.470+08:00 and status AVAILABLE 2024-03-13 15:57:39.479 INFO 1433467 --- [pool-Check host task for agent11,898edbfdd3684000,a43d1d8c3517] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:57:39.479 INFO 1433467 --- [pool-Check host task for agent11,898edbfdd3684000,a43d1d8c3517] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:57:39.479 INFO 1433467 --- [pool-Check host task for agent11,898edbfdd3684000,a43d1d8c3517] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:57:39.483 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:57:39.484 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:57:39.483792096+08:00 2024-03-13 15:57:39.484 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:57:39.484 INFO 1433467 --- [pool-Check host task for agent17,898edbfdd3684000,103929c2d82e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:57:39.484 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:57:39.484+08:00 2024-03-13 15:57:39.487 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:57:39.484+08:00 and status AVAILABLE 2024-03-13 15:57:39.488 INFO 1433467 --- [pool-Check host task for agent24,898edbfdd3684000,fdd4c0fc5248] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:57:39.524 WARN 1433467 --- [pool-Check host task for clockdiff22,9a05e8bf9c114c78,9336a8c9b851] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.525 ERROR 1433467 --- [pool-Check host task for clockdiff22,9a05e8bf9c114c78,9336a8c9b851] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:57:39.525 WARN 1433467 --- [pool-Check host task for clockdiff22,9a05e8bf9c114c78,9336a8c9b851] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.525 WARN 1433467 --- [pool-Check host task for clockdiff6,9a05e8bf9c114c78,d9a5caea90b5] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.525 ERROR 1433467 --- [pool-Check host task for clockdiff6,9a05e8bf9c114c78,d9a5caea90b5] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:57:39.525 WARN 1433467 --- [pool-Check host task for clockdiff6,9a05e8bf9c114c78,d9a5caea90b5] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.537 INFO 1433467 --- [pool-Check host task for agent24,898edbfdd3684000,fdd4c0fc5248] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:57:39.537 INFO 1433467 --- [pool-Check host task for agent24,898edbfdd3684000,fdd4c0fc5248] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:57:39.537 INFO 1433467 --- [pool-Check host task for agent24,898edbfdd3684000,fdd4c0fc5248] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:57:39.538 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:57:39.538 INFO 1433467 --- [pool-Check host task for agent17,898edbfdd3684000,103929c2d82e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:57:39.538 INFO 1433467 --- [pool-Check host task for agent17,898edbfdd3684000,103929c2d82e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:57:39.538 INFO 1433467 --- [pool-Check host task for agent17,898edbfdd3684000,103929c2d82e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:57:39.541 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:57:39.542 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:57:39.542 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:57:39.543 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:57:39.543 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:57:39.544 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:57:39.549 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:57:39.550 WARN 1433467 --- [pool-scanner-1609805699-1,4ee7d417e990464f,a9efa5053b0a] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:57:39.550 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:57:39.554 WARN 1433467 --- [pool-Check host task for clockdiff13,9a05e8bf9c114c78,9595359e5c26] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.554 ERROR 1433467 --- [pool-Check host task for clockdiff13,9a05e8bf9c114c78,9595359e5c26] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:57:39.554 WARN 1433467 --- [pool-Check host task for clockdiff13,9a05e8bf9c114c78,9595359e5c26] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.555 WARN 1433467 --- [pool-Check host task for clockdiff24,9a05e8bf9c114c78,0b9c83a7e239] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.555 WARN 1433467 --- [pool-Check host task for clockdiff29,9a05e8bf9c114c78,e827ca571f2a] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.583 ERROR 1433467 --- [pool-Check host task for clockdiff24,9a05e8bf9c114c78,0b9c83a7e239] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:57:39.581 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:39.583 ERROR 1433467 --- [pool-Check host task for clockdiff29,9a05e8bf9c114c78,e827ca571f2a] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:57:39.583 WARN 1433467 --- [pool-Check host task for clockdiff24,9a05e8bf9c114c78,0b9c83a7e239] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.583 WARN 1433467 --- [pool-Check host task for clockdiff2,9a05e8bf9c114c78,abd12b5eb054] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.583 ERROR 1433467 --- [pool-Check host task for clockdiff2,9a05e8bf9c114c78,abd12b5eb054] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:57:39.583 WARN 1433467 --- [pool-Check host task for clockdiff29,9a05e8bf9c114c78,e827ca571f2a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.583 WARN 1433467 --- [pool-Check host task for clockdiff8,9a05e8bf9c114c78,a5f644a3af26] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.584 WARN 1433467 --- [pool-Check host task for clockdiff2,9a05e8bf9c114c78,abd12b5eb054] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.582 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:39.584 ERROR 1433467 --- [pool-Check host task for clockdiff8,9a05e8bf9c114c78,a5f644a3af26] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:57:39.584 WARN 1433467 --- [pool-Check host task for clockdiff8,9a05e8bf9c114c78,a5f644a3af26] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:57:39.582 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:57:39.582 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:57:39.585 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:57:39.585 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:57:39.585 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:57:39.586 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:57:39.589 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:39.598 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:39.599 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:39.599 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:57:39.601 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:39.603 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:39.603 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.603 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.603 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.604 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:57:39.607 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:57:39.615 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:57:39.619 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:57:39.626 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:57:39.630 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:57:39.631 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:57:39.631 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:57:39.635 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:57:39.637 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:57:39.639 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:57:39.639 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:57:39.643 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:57:39.645 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:39.647 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:39.648 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:39.648 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.649 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.649 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.649 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:57:39.649 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:39.650 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:39.650 WARN 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:57:39.651 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.651 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.651 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:39.652 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:57:39.653 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:57:39.653 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:57:39.658 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:57:39.660 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:39.661 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:39.661 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:57:39.661 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:57:39.662 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:57:39.667 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:57:39.668 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:57:39.673 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:57:39.675 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:39.714 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:39.715 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:39.715 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:39.716 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.716 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.716 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.716 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:57:39.717 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:57:39.722 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:57:39.724 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:57:39.724 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:57:39.725 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:57:39.728 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:57:39.729 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:57:39.729 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:57:39.729 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:57:39.730 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:57:39.730 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:57:39.730 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:57:39.735 INFO 1433467 --- [pool-Check host task for host53,70e3aa2894294a65,f9eb021af8fc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:57:39.735 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:57:39.736 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:57:39.738 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:57:39.738 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:57:39.739 INFO 1433467 --- [pool-Check host task for host48,70e3aa2894294a65,179d1e010b81] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:57:39.744 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:39.750 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:39.754 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:39.755 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:57:39.757 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:39.758 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:57:39.769 WARN 1433467 --- [ForkJoinPool-1386595-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : Failed to get ob process info, errMsg:[AgentClient]:http request is failed, response:Cannot get process info of open /proc/9805/status: no such file or directory, reason: %!v(MISSING) 2024-03-13 15:57:39.774 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:57:39.779 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:39.779 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.781 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:57:39.781 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.782 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:57:39.786 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:57:39.786 INFO 1433467 --- [ForkJoinPool-1386595-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316540069000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:57:39.787 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:57:39.788 INFO 1433467 --- [pool-Check host task for host13,70e3aa2894294a65,2f9652a8d30c] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:57:39.793 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:57:39.794 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:57:39.797 INFO 1433467 --- [pool-Check host task for host46,70e3aa2894294a65,86de81037841] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:57:39.803 INFO 1433467 --- [pool-Check host task for host51,70e3aa2894294a65,3a514726264b] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:57:39.805 INFO 1433467 --- [pool-Check host task for host54,70e3aa2894294a65,a83328cf2b51] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:57:39.809 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:57:39.816 INFO 1433467 --- [pool-Check host task for host44,70e3aa2894294a65,7730eaa0035a] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:57:39.826 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:39.826 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.827 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.827 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.827 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:57:39.848 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:39.849 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.849 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.849 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.849 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:57:39.852 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.852 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.852 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:57:39.855 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:57:39.862 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:39.862 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:57:39.869 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:39.869 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:57:39.876 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:39.876 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:57:39.883 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:39.883 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:57:39.888 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:39.888 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:57:39.900 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:57:39.918 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:39.919 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.919 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:39.919 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:39.920 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:57:39.923 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:57:39.925 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806557,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:39.929 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:57:39.934 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:57:39.936 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:57:39.961 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:57:39.997 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:57:40.019 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:57:40.020 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:40.020 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:57:40.020 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:57:40.021 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:57:40.038 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:57:40.038 INFO 1433467 --- [ForkJoinPool-1386595-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:57:40.212 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:40.213 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:41.925 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806580,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:43.213 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:43.214 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:43.925 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806603,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:45.926 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806626,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:46.111 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806629,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:57:46.214 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:46.215 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:46.585 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : Starting SQL_PERF_DIAGNOSER, Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.588 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : SQL_PERF_DIAGNOSER : for [Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null)] 2024-03-13 15:57:46.606 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : ROW_LOCK_CONTENTION_HIGH found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.606 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : PERFORMANCE_DEGRADATION is disabled or lacking of config 2024-03-13 15:57:46.625 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : EXECUTION_SPIKE found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.640 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : CPU_TIME_PROPORTION_HIGH found out 6 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.683 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : DML_WITH_UNNECESSARY_PARTITION found out 0 diagnose targets Pair(left=Tenant(id=1, name=sys, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone2;zone1;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.688 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : ROW_LOCK_CONTENTION_HIGH found out 0 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.701 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : PERFORMANCE_DEGRADATION found out 3 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.728 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : EXECUTION_SPIKE found out 0 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.736 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : CPU_TIME_PROPORTION_HIGH found out 12 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.761 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:57:46.760226+08:00, lastProcessTime=2024-03-13T15:57:36.774316+08:00 2024-03-13 15:57:46.793 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : DML_WITH_UNNECESSARY_PARTITION found out 0 diagnose targets Pair(left=Tenant(id=2, name=tenant1, clusterName=obcluster1, obClusterId=1, clusterId=1, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:46:30+08:00, primaryZone=zone1;zone2;zone3, zoneList=zone1;zone2;zone3, locality=FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.794 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : ROW_LOCK_CONTENTION_HIGH found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.794 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : PERFORMANCE_DEGRADATION is disabled or lacking of config 2024-03-13 15:57:46.795 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : EXECUTION_SPIKE found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.796 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : CPU_TIME_PROPORTION_HIGH found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.797 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : DML_WITH_UNNECESSARY_PARTITION found out 0 diagnose targets Pair(left=Tenant(id=3, name=sys, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=RANDOM, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.797 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : ROW_LOCK_CONTENTION_HIGH found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.798 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : PERFORMANCE_DEGRADATION found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.799 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : EXECUTION_SPIKE found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.799 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : CPU_TIME_PROPORTION_HIGH found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.800 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.SqlPerfDiagnoseService : DML_WITH_UNNECESSARY_PARTITION found out 0 diagnose targets Pair(left=Tenant(id=4, name=ocp, clusterName=obcluster, obClusterId=1, clusterId=2, arbitrationServiceId=null, clusterType=PRIMARY, clusterStatus=RUNNING, syncStatus=VALID, mode=MYSQL, createTime=2023-08-24T15:57:23+08:00, primaryZone=zone1, zoneList=zone1, locality=FULL{1}@zone1, status=NORMAL, operatingTaskInstanceId=null, statusAnalysis=null, locked=false, readonly=false, obVersion=4.2.1.4, description=null, obTenantId=1002, sqlAuditEnabled=null, arbitrationStatus=DISABLED, role=PRIMARY, logTransportMode=null, ocpClusterView=null, basicPrimaryTenant=null, basicStandbyTenantList=null), right=null), Interval{start=2024-03-13T15:51:46.366, end=2024-03-13T15:56:46.58} 2024-03-13 15:57:46.800 INFO 1433467 --- [pool-13-thread-1,,] c.o.o.p.s.d.i.AbstractDiagnoseService : SQL_PERF_DIAGNOSER, Found out 5 dubious sqls 2024-03-13 15:57:46.857 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:57:46.760226+08:00 2024-03-13 15:57:47.029 INFO 1433467 --- [pool-ocp-schedules-15,ecdb5f29f56142c7,fd81d4a78c84] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:57:47.908 INFO 1433467 --- [pool-clean-time-series-data15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:57:47.925 INFO 1433467 --- [pool-print-time-series-interval15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316667: {1710316560=1394, 1710316575=1394, 1710316590=1394, 1710316605=1394, 1710316620=1394, 1710316635=1394, 1710316650=1394, 1710316665=60}, tsDataMapSize: 1122288, readyDataCacheSize: 8288 2024-03-13 15:57:47.926 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806650,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:47.948 INFO 1433467 --- [pool-clean-time-series-data60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:57:47.949 INFO 1433467 --- [pool-print-time-series-interval60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316667: {1710316500=34, 1710316560=32, 1710316620=6}, tsDataMapSize: 17768, readyDataCacheSize: 18488 2024-03-13 15:57:48.734 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,f3dd8c884afd43e2,29b3d993d7ff] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.143, traceId=f3dd8c884afd43e2, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:57:48.745 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,f3dd8c884afd43e2,29b3d993d7ff] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.143, traceId=f3dd8c884afd43e2, duration=12 ms] 2024-03-13 15:57:49.215 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:49.216 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:49.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806665,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:57:49.319 WARN 1433467 --- [pool-scanner-1609806256-1,caca7ff146d3425c,fb050c9b7fae] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:57:49.406 INFO 1433467 --- [alarm-detect-5,097986d9aacc4c3e,93d4255cdf96] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:57:49.408 INFO 1433467 --- [alarm-detect-5,097986d9aacc4c3e,93d4255cdf96] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:57:49.444 INFO 1433467 --- [alarm-detect-9,b80b257f9d4746b3,ad8c566d653e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316635 ~ 1710316659 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316650=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:57:49.444 INFO 1433467 --- [alarm-detect-9,b80b257f9d4746b3,ad8c566d653e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316635 ~ 1710316659 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316650=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:57:49.444 INFO 1433467 --- [alarm-detect-9,b80b257f9d4746b3,ad8c566d653e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316635 ~ 1710316659 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316650=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:57:49.444 INFO 1433467 --- [alarm-detect-9,b80b257f9d4746b3,ad8c566d653e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316635 ~ 1710316659 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316650=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:57:49.927 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806674,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:51.927 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806697,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:52.216 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:52.218 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:53.248 INFO 1433467 --- [pool-ocp-schedules-5,cdcb1651d3434364,3684fd62c31f] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:57:53.254 INFO 1433467 --- [pool-ocp-schedules-5,cdcb1651d3434364,3684fd62c31f] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:57:53.254 INFO 1433467 --- [pool-ocp-schedules-5,cdcb1651d3434364,3684fd62c31f] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:57:53.928 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806720,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:55.218 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:55.221 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:55.737 INFO 1433467 --- [pool-ocp-schedules-6,f17b4af12af54a3b,40ecf5f402b6] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2024-03-13 15:57:55.737 INFO 1433467 --- [pool-ocp-schedules-6,f17b4af12af54a3b,40ecf5f402b6] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=609459010050985984 2024-03-13 15:57:55.748 INFO 1433467 --- [pool-ocp-schedules-6,f17b4af12af54a3b,40ecf5f402b6] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2024-03-13 15:57:55.749 INFO 1433467 --- [pool-ocp-schedules-6,f17b4af12af54a3b,40ecf5f402b6] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624101396131614721 2024-03-13 15:57:55.756 INFO 1433467 --- [pool-ocp-schedules-6,f17b4af12af54a3b,40ecf5f402b6] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2024-03-13 15:57:55.756 INFO 1433467 --- [pool-ocp-schedules-6,f17b4af12af54a3b,40ecf5f402b6] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624425259050733570 2024-03-13 15:57:55.928 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806743,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:56.111 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806745,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:57:56.614 INFO 1433467 --- [pool-Check obproxy15,4c551e9a464f4a03,1845a35ea999] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:57:56.615 INFO 1433467 --- [pool-Check obproxy12,4c551e9a464f4a03,e79d073c232b] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:57:56.615 INFO 1433467 --- [pool-Check obproxy10,4c551e9a464f4a03,71c8beb59c4a] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:57:56.617 INFO 1433467 --- [pool-Check obproxy5,4c551e9a464f4a03,988f85359cab] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:57:56.747 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:57:56.747087+08:00, lastProcessTime=2024-03-13T15:57:46.760226+08:00 2024-03-13 15:57:56.844 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:57:56.747087+08:00 2024-03-13 15:57:57.035 INFO 1433467 --- [pool-ocp-schedules-9,09c2867517ca4af4,478e5b7dd5c2] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:57:57.928 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806766,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:57:58.221 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:57:58.222 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:57:59.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806782,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:57:59.280 INFO 1433467 --- [alarm-detect-20,e989b92f34fd4cea,de5f4f0d4642] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:57:59.282 INFO 1433467 --- [alarm-detect-11,075f136dd9364f17,806f5bbce95e] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:57:59.282 INFO 1433467 --- [alarm-detect-9,416b187a166047d3,20d9a00b97bb] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:57:59.284 INFO 1433467 --- [alarm-detect-20,e989b92f34fd4cea,de5f4f0d4642] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:57:59.286 INFO 1433467 --- [alarm-detect-9,416b187a166047d3,20d9a00b97bb] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:57:59.287 INFO 1433467 --- [alarm-detect-11,075f136dd9364f17,806f5bbce95e] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:57:59.318 WARN 1433467 --- [pool-scanner-1609806906-1,1ea4a9369a1f4d4e,29da0a7a6d7e] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:57:59.421 INFO 1433467 --- [alarm-detect-10,1c3dd2aac1bf4112,2bd51af84992] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316650 ~ 1710316669 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316665=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:57:59.421 INFO 1433467 --- [alarm-detect-10,1c3dd2aac1bf4112,2bd51af84992] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316650 ~ 1710316669 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316665=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:57:59.421 INFO 1433467 --- [alarm-detect-10,1c3dd2aac1bf4112,2bd51af84992] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316650 ~ 1710316669 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316665=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:57:59.421 INFO 1433467 --- [alarm-detect-10,1c3dd2aac1bf4112,2bd51af84992] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316650 ~ 1710316669 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316665=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:57:59.421 INFO 1433467 --- [alarm-detect-10,1c3dd2aac1bf4112,2bd51af84992] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316650 ~ 1710316669 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316665=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:57:59.421 INFO 1433467 --- [alarm-detect-10,1c3dd2aac1bf4112,2bd51af84992] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316650 ~ 1710316669 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316665=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:57:59.421 INFO 1433467 --- [alarm-detect-10,1c3dd2aac1bf4112,2bd51af84992] c.o.o.a.core.detect.AlarmRuleDetector : detectByMetric done, alarmType=host_ntp_service_not_exist, needSendingCount=1, detectAlarmsCount=1, matchedTargets=[alarm_template_id=3:host=10.10.10.148-2, alarm_template_id=3:host=10.10.10.147-2, alarm_template_id=3:host=10.10.10.146-2, alarm_template_id=3:host=10.10.10.145-2, alarm_template_id=3:host=10.10.10.144-2, alarm_template_id=3:host=10.10.10.143-2] 2024-03-13 15:57:59.441 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806782,,] c.o.o.a.c.t.DistributedAlarmDetectTask : Refresh detector factory. 2024-03-13 15:57:59.929 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806790,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:00.004 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:57:01.0, nextRun=2024-03-13 15:58:00.0, currentDate=Wed Mar 13 15:58:00 CST 2024 2024-03-13 15:58:00.004 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999981, name=Collect all tenants major compactions, templateName=CollectTenantCompactionSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:57:01.0, nextRunTime=2024-03-13 15:58:00.0} 2024-03-13 15:58:00.010 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all tenants major compactions, nodeCount=1, taskCount=1 2024-03-13 15:58:00.011 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:00.011 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all tenants major compactions, taskDefinitionId=999981, state=RUNNING, operation=EXECUTE} 2024-03-13 15:58:00.021 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:57:01.0, nextRun=2024-03-13 15:58:00.0, currentDate=Wed Mar 13 15:58:00 CST 2024 2024-03-13 15:58:00.021 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999982, name=Collect SlowSql to SqlAggregateData for alarm, templateName=CollectSlowSqlSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:57:01.0, nextRunTime=2024-03-13 15:58:00.0} 2024-03-13 15:58:00.029 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect SlowSql to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:58:00.029 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:00.029 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect SlowSql to SqlAggregateData for alarm, taskDefinitionId=999982, state=RUNNING, operation=EXECUTE} 2024-03-13 15:58:00.035 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:57:01.0, nextRun=2024-03-13 15:58:00.0, currentDate=Wed Mar 13 15:58:00 CST 2024 2024-03-13 15:58:00.035 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999983, name=Collect trans stat to SqlAggregateData for alarm, templateName=CollectTransSatatSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:57:01.0, nextRunTime=2024-03-13 15:58:00.0} 2024-03-13 15:58:00.040 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect trans stat to SqlAggregateData for alarm, nodeCount=1, taskCount=1 2024-03-13 15:58:00.040 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:00.040 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect trans stat to SqlAggregateData for alarm, taskDefinitionId=999983, state=RUNNING, operation=EXECUTE} 2024-03-13 15:58:00.045 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:57:01.0, nextRun=2024-03-13 15:58:00.0, currentDate=Wed Mar 13 15:58:00 CST 2024 2024-03-13 15:58:00.045 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999984, name=Get agent alarms, templateName=GetAgentAlarmsSchedule, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:57:01.0, nextRunTime=2024-03-13 15:58:00.0} 2024-03-13 15:58:00.048 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Get agent alarms, nodeCount=1, taskCount=1 2024-03-13 15:58:00.048 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:00.048 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Get agent alarms, taskDefinitionId=999984, state=RUNNING, operation=EXECUTE} 2024-03-13 15:58:00.054 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:01.0, nextRun=2024-03-13 15:58:00.0, currentDate=Wed Mar 13 15:58:00 CST 2024 2024-03-13 15:58:00.054 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999992, name=Sync root service list, templateName=SyncRsListSchedule, scheduleRule=0 */2 * * * ?, lastRunTime=2024-03-13 15:56:01.0, nextRunTime=2024-03-13 15:58:00.0} 2024-03-13 15:58:00.057 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Sync root service list, nodeCount=1, taskCount=2 2024-03-13 15:58:00.057 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:00.057 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:00.057 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Sync root service list, taskDefinitionId=999992, state=RUNNING, operation=EXECUTE} 2024-03-13 15:58:00.064 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:57:31.0, nextRun=2024-03-13 15:58:00.0, currentDate=Wed Mar 13 15:58:00 CST 2024 2024-03-13 15:58:00.064 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:57:31.0, nextRunTime=2024-03-13 15:58:00.0} 2024-03-13 15:58:00.066 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:58:00.066 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:00.066 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:58:00.566 INFO 1433467 --- [pool-prometheus-minute-schedule-1,33a7df0b820a45a5,840035fdce3c] c.o.ocp.monitor.OcpMonitorManager : Next exec sec=1710316740, interval=60 2024-03-13 15:58:00.567 INFO 1433467 --- [pool-prometheus-cache-monitor1,9ad5ae4da2b84d9d,7e83e1e11828] c.o.o.m.s.OcpCacheMonitorServiceImpl : OcpCacheMonitor, cacheInfo={"secondReadCacheSize":12077,"minuteWriteQueue":0,"minuteReadCacheSize":12328,"secondWriteCacheSize":3323,"seriesIdKeySize":67930,"secondWriteQueue":0,"minuteWriteCacheSize":12404} 2024-03-13 15:58:00.567 INFO 1433467 --- [pool-prometheus-cache-monitor1,9ad5ae4da2b84d9d,7e83e1e11828] c.o.o.m.s.OcpCacheMonitorServiceImpl : MetricSearchStats:{go_memory_classes_heap_stacks_bytes=15, ob_sysstat_40012=19, ob_server_resource_memory_assigned_bytes=13, ob_sysstat_40011=19, ob_sysstat_40010=19, ob_unit_num=13, ob_database_disk_required_size=35, go_memstats_mspan_sys_bytes=15, go_memstats_stack_sys_bytes=15, ob_tenant_request_queue_total_size=18, jvm_gc_memory_allocated_bytes_total=3, ob_system_event_total_waits=24, ob_tenant_disk_v4_log_disk_size=18, ob_sysstat_40019=19, observer_fd_count=13, ob_sysstat_40018=19, processor_process_metrics_total=21, jvm_gc_max_data_size_bytes=3, go_memstats_frees_total=15, sql_audit_input_collect_data_jumped_total=8, host_available=64, mysql_output_write_sql_failed_count=17, ob_database_disk_data_size=34, ob_system_event_detail_total_timeouts=763, ob_tenant_memstore_freeze_cnt=14, ob_latch_sleep_total=312, ob_tenant_unit_min_memory=25, ob_sysstat_60087=19, ob_memstore_total_bytes=20, metric_cardinality_count=12, process_resident_memory_bytes=15, jvm_threads_states_threads=8, ob_cache_size_bytes=34, ob_tenant_unit_max_memory=25, input_collect_metrics_total=26, ob_sysstat_60083=19, ob_sysstat_50039=19, ob_sysstat_50038=19, node_disk_io_time_weighted_seconds_total=33, process_open_fds=15, ob_sysstat_50037=19, process_memory_rss_bytes=33, ob_server_resource_cpu_assigned_percent=13, ob_sysstat_50030=19, go_memory_classes_profiling_buckets_bytes=15, go_memstats_heap_inuse_bytes=15, ob_sysstat_40118=19, ob_sysstat_40117=19, ob_sysstat_40116=19, node_memory_memfree_bytes=26, ob_sysstat_40005=19, process_exists=43, ob_sysstat_40004=19, ob_sysstat_40003=19, ob_sysstat_40002=19, ob_sysstat_40001=19, go_gc_duration_seconds=20, go_memory_classes_metadata_mspan_inuse_bytes=15, ob_server_resource_disk_bytes=13, ob_sysstat_40000=19, odp_current_session=26, process_thread_count=33, ob_sysstat_50040=19, ob_sysstat_40009=19, ob_sysstat_40008=19, ob_disk_free_bytes=13, node_filesystem_avail_bytes=72, ob_sysstat_40007=19, ob_sysstat_40006=19, ob_backup_storage_capacity_bytes=650, ob_unit_config_max_iops=21, ob_plan_cache_memory_bytes=18, node_network_transmit_errs_total=28, go_gc_duration_seconds_count=15, monagent_oceanbase_plan_explain_size=4, ob_sysstat_50011=19, ob_sysstat_50010=19, node_disk_read_bytes_total=33, ob_compaction_is_suspended=19, mysql_output_write_sql_milliseconds_summary_count=17, go_memstats_lookups_total=15, mysql_output_write_sql_count=17, process_cpu_percent=33, ob_sysstat_60057=19, host_agent_process_status=18, ob_server_resource_cpu=13, ob_tenant_server_data_index_row_count=20, go_memstats_alloc_bytes=15, go_sched_gomaxprocs_threads=15, node_cpu_seconds_total_native=38, ob_sysstat_140002=19, ob_sysstat_140005=19, ob_sysstat_140006=19, ob_sysstat_140003=19, ob_tenant_resource_min_memory=18, ob_memstore_freeze_times=20, ob_sysstat_50022=19, jvm_gc_pause_seconds_count=8, ob_sysstat_50021=19, jvm_gc_overhead_percent=3, go_memory_classes_heap_unused_bytes=15, ob_cluster_status_check=13, jvm_memory_committed_bytes=10, node_net_bandwidth_bps=29, monagent_oceanbase_db_pool_idle=4, ob_tenant_server_data_index_data_size=20, go_gc_heap_allocs_by_size_bytes_total_count=15, go_memstats_heap_idle_bytes=15, go_gc_cycles_forced_gc_cycles_total=15, go_gc_pauses_seconds_total_bucket=26, ob_server_resource_memory_bytes=13, ob_tenant_assigned_mem_assigned=18, ob_tenant_memstore_minor_freeze_times=14, ob_tenant_server_required_size=22, ob_sysstat_60031=19, process_fd_count=33, ob_tenant_unit_max_cpu=25, ob_latch_wait_seconds_total=312, go_memory_classes_metadata_mcache_free_bytes=15, node_memory_buffers_bytes=26, ob_sysstat_50009=19, go_memstats_heap_released_bytes=15, node_network_receive_bytes_total=28, ob_sysstat_50005=19, ob_sysstat_50008=19, ob_sysstat_50001=19, ob_sysstat_50004=19, exposer_request_latencies_count=6, ob_sysstat_50000=19, ob_session_active_num=18, mysql_output_context_timeout_discard_metrics_count=9, go_memory_classes_metadata_mcache_inuse_bytes=15, node_disk_read_time_seconds_total=33, ob_sysstat_60021=19, ob_sysstat_60020=19, ob_sysstat_60024=19, ob_compaction_frozen_version=19, ob_sysstat_60023=19, metric_count=15, ob_sysstat_60022=19, obproxysys_connectable=1556, ob_tenant_assigned_cpu_assigned=18, mysql_output_write_sql_retry_count=17, ob_tenant_unit_min_cpu=25, ob_unit_config_max_cpu=21, http_request_milliseconds_summary_count=75, monagent_oceanbase_audit_timeinfo_cache_size=4, http_request_milliseconds_summary=78, ob_latch_immediate_miss_total=312, ob_latch_spin_get_total=312, ob_log_level_count=17, monagent_pipeline_report_metrics_total=16, exposer_transferred_bytes_total=6, ob_tenant_context_memory_count=841, node_disk_reads_completed_total=33, ob_sysstat_60019=19, ob_sysstat_60003=19, ob_sysstat_60002=19, ob_sysstat_60001=19, ob_sysstat_60000=19, ob_latch_miss_total=312, go_sched_latencies_seconds_count=15, ocp_meta_db_health=5, ob_disk_total_bytes=13, go_memstats_heap_objects=15, go_memstats_mcache_inuse_bytes=15, ob_sysstat_60005=19, ob_sysstat_60004=19, go_memory_classes_other_bytes=15, ob_memstore_active_bytes=20, go_memstats_alloc_bytes_total=15, ob_tenant_server_partition_replica_count=21, process_count=34, ob_system_event_detail_total_waits=763, odp_used_connections=25, backup_storage_capacity_used_percent=8, go_gc_heap_frees_objects_total=15, go_memory_classes_metadata_mspan_free_bytes=15, ob_disk_log_used_bytes=13, ob_compaction_is_error=19, go_gc_limiter_last_enabled_gc_cycle=15, monagent_pipeline_execute_total=24, node_filesystem_files=86, process_start_time_seconds=15, go_memstats_heap_alloc_bytes=15, ob_tenant_resource_max_cpu=18, ntp_service_available=26, go_memstats_mspan_inuse_bytes=15, mysql_output_write_sql_milliseconds_summary=20, go_gc_heap_frees_by_size_bytes_total_count=15, monagent_pipeline_execute_seconds_total=24, ob_sysstat_80001=19, go_memstats_sys_bytes=15, go_memory_classes_heap_free_bytes=15, system_load_average_1m=3, monagent_oceanbase_audit_stat_cache_size=4, ob_tenant_memory_used_bytes=22, obproxy_server_max_connections=7, ob_sysstat_80002=19, go_memstats_gc_cpu_fraction=15, ob_sysstat_80003=19, monagent_oceanbase_audit_cache_size=4, ob_tenant_context_memory_used_bytes=841, ob_sysstat_80007=19, node_disk_writes_completed_total=33, ob_sysstat_80008=19, ob_sysstat_80009=19, go_sched_goroutines_goroutines=15, go_memory_classes_metadata_other_bytes=15, go_gc_cycles_total_gc_cycles_total=15, system_cpu_usage=3, process_cpu_usage=3, node_disk_written_bytes_total=33, monagent_oceanbase_ash_cache_size=4, go_goroutines=15, ob_sysstat_80023=19, exposer_request_latencies_sum=6, ob_unit_config_min_cpu=21, process_virtual_memory_max_bytes=15, go_gc_heap_frees_bytes_total=15, ob_tenant500_memory_hold_bytes=13, exposer_scrapes_total=6, go_gc_heap_allocs_by_size_bytes_total_bucket=27, ob_sysstat_80025=19, ob_sysstat_80026=19, oceanbase_connectivity=15, node_memory_sreclaimable_bytes=26, go_sched_latencies_seconds_sum=15, tcp_retrans=25, exporter_export_metrics_bytes_total=26, node_filesystem_size_bytes=72, http_server_requests_seconds_count=203, ob_compaction_frozen_time=19, monagent_input_sql_duration_seconds_sum=55, log_stream_degraded_count=18, ob_plan_cache_hit_total=18, node_filesystem_readonly=72, ob_sysstat_80040=19, ob_sysstat_80041=19, io_await=52, go_gc_heap_allocs_objects_total=15, go_memstats_mallocs_total=15, ob_unit_config_max_memory_bytes=21, go_gc_stack_starting_size_bytes=15, io_util=52, ob_sysstat_200002=19, ob_sysstat_200001=19, ob_tenant_disk_v4_data_size=18, go_sched_latencies_seconds_bucket=26, ob_tenant_assigned_cpu_total=18, ob_tenant_server_available_disk_free=19, node_disk_write_time_seconds_total=33, cpu_count=25, output_write_metrics_total=9, ob_cluster_sync=15, go_gc_heap_objects_objects=15, odp_transaction_total=16, ob_system_event_detail_time_waited_us=763, process_max_fds=15, http_status_count=20, ob_sysstat_80057=19, ocp_alarm_detect_duration_ms_count=115, go_gc_heap_allocs_by_size_bytes_total_sum=15, go_memstats_mcache_sys_bytes=15, ob_tenant_disk_v4_log_size=18, go_cgo_go_to_c_calls_calls_total=15, ocp_alarm_detect_duration_ms_sum=115, jvm_threads_live_threads=3, node_memory_cached_bytes=26, ob_table_num=16, ob_sysstat_10000=19, ob_sysstat_10006=19, ob_sysstat_10005=19, ob_sysstat_10002=19, ob_sysstat_10001=19, go_memstats_last_gc_time_seconds=15, ob_sysstat_10004=19, ob_sysstat_10003=19, monitor_exporter_avaliable=62, log_tailer_count=15, ob_server_num=25, ob_session_all_num=18, ob_system_event_time_waited=24, ob_tenant_resource_min_cpu=18, monagent_plugin_execute_seconds_total=39, go_gc_cycles_automatic_gc_cycles_total=15, go_gc_heap_allocs_bytes_total=15, go_memstats_gc_sys_bytes=15, ob_latch_get_total=312, go_info=16, log_tailer_processed_file_count=21, node_load15=24, ob_disk_log_assigned_bytes=13, go_memory_classes_heap_released_bytes=15, jvm_memory_max_bytes=10, ob_unit_config_min_iops=21, ob_sysstat_30080=19, ob_sysstat_30082=19, ob_sysstat_30081=19, ob_sysstat_30084=19, ob_sysstat_30083=19, ob_sysstat_30086=19, ob_sysstat_30085=19, ob_tenant_log_disk_total_bytes=18, ob_memstore_freeze_trigger_bytes=20, ob_waitevent_wait_seconds_total=18, ob_server_stopped_duration_seconds=15, go_memstats_next_gc_bytes=15, node_filesystem_files_free=86, ob_memtable_max_snapshot_duration_seconds=18, monagent_oceanbase_audit_unique_key_cache_size=4, process_virtual_memory_bytes=15, ob_sysstat_20002=19, net_connectivity=15, ob_sysstat_20001=19, monagent_input_sql_duration_seconds_bucket=65, ob_latch_immediate_get_total=312, node_cpu_seconds_total=32, ob_sysstat_20000=19, ob_index_error_num=13, ob_tenant_server_available_disk_total=19, ob_tenant_memory_hold_bytes=22, ob_sysstat_30066=19, go_memstats_buck_hash_sys_bytes=15, mysql_output_write_sql_milliseconds_summary_sum=17, ob_tenant_disk_total_size=18, odp_sql_request_total=53, mysql_output_write_task_size=17, exporter_export_metrics_total=26, ob_disk_log_total_bytes=13, system_cpu_count=3, ob_sysstat_130004=19, process_uptime_seconds=33, node_ntp_offset_seconds=25, go_memstats_heap_sys_bytes=15, ob_sysstat_140012=19, ob_sysstat_140013=19, jvm_gc_pause_seconds_max=8, ob_tenant_request_queue_large_size=18, ob_sysstat_130000=19, go_memstats_other_sys_bytes=15, ob_sysstat_130001=19, jvm_memory_used_bytes=10, ob_sysstat_130002=19, go_threads=15, http_request_milliseconds_summary_sum=75, http_server_requests_seconds_sum=203, ob_tenant_assigned_mem_total=18, node_memory_memtotal_bytes=26, node_network_transmit_drop_total=28, go_gc_pauses_seconds_total_count=15, exposer_request_latencies=9, ob_waitevent_wait_total=18, go_gc_heap_goal_bytes=15, ob_tenant_log_disk_used_bytes=18, jvm_gc_memory_promoted_bytes_total=3, odp_sql_cost_total=51, ob_sysstat_30011=19, ob_sysstat_30010=19, ob_sysstat_30013=19, ob_sysstat_30012=19, ob_tenant500_memory_used_bytes=13, node_network_transmit_bytes_total=28, node_network_receive_drop_total=28, go_gc_heap_tiny_allocs_objects_total=15, go_gc_duration_seconds_sum=15, ob_server_resource_memory_assigned_percent=13, jvm_gc_pause_seconds_sum=8, go_memory_classes_heap_objects_bytes=15, node_network_receive_errs_total=28, observer_ulimit_max_fd_count=13, ob_plan_cache_access_total=18, ob_tenant_server_row_count=22, monagent_oceanbase_plan_cache_size=4, ob_tenant_server_data_size=22, monagent_oceanbase_audit_sample_cache_size=4, go_gc_heap_frees_by_size_bytes_total_sum=15, ob_sysstat_40030=19, node_load1=24, process_cpu_seconds_total=15, ob_sysstat_120001=19, go_memory_classes_total_bytes=15, ob_sysstat_120002=19, ob_sysstat_120000=19, ob_tenant_resource_max_memory=18, ob_sysstat_120009=19, go_memstats_stack_inuse_bytes=15, ob_sysstat_120008=19, node_load5=24, ob_sysstat_120005=19, go_memory_classes_os_stacks_bytes=15, ob_sysstat_120006=19, jvm_threads_peak_threads=3, monagent_input_sql_duration_seconds_count=55, ob_tenant_context_memory_hold_bytes=841, obproxy_memory_limit_bytes=7, ob_compaction_last_version=19, ob_sysstat_30000=19, ob_server_resource_cpu_assigned=13, ob_sysstat_30002=19, ob_sysstat_30001=19, monagent_oceanbase_plan_timeinfo_cache_size=4, monagent_plugin_execute_total=39, ob_sysstat_30006=19, ob_sysstat_30005=19, ob_sysstat_30008=19, ob_sysstat_30007=19, ob_sysstat_30009=19, go_gc_heap_frees_by_size_bytes_total_bucket=27, obproxy_client_max_connections=7, ob_compaction_current_timestamp=19, ob_clog_max_sync_delay_seconds=20, go_gc_pauses_seconds_total_sum=15, ob_compaction_start_time=19} 2024-03-13 15:58:00.572 INFO 1433467 --- [pool-prometheus-minute-schedule-1,33a7df0b820a45a5,840035fdce3c] c.o.ocp.monitor.OcpMonitorManager : Collect interval metrics, interval=60, exporterCount=18, collectTaskCount=18 2024-03-13 15:58:00.572 INFO 1433467 --- [pool-prometheus-minute-schedule-1,33a7df0b820a45a5,840035fdce3c] c.o.ocp.monitor.OcpMonitorManager : Schedule interval metric too slow, from=1710316680, count=18, elapsed=1708606363892 2024-03-13 15:58:00.931 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016769, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:00.946 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016770, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:00.952 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,16db867fedfa] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016769, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014416, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:00.929+08:00, sub_task_instance_name=Collect all tenants major compactions, sub_task_instance_id=10016769}, listMap={}}, executor=10.10.10.148 2024-03-13 15:58:00.952 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,16db867fedfa] o.s.t.b.t.CollectAllTenantCompactionTask : Begin to collect all tenants major compaction info 2024-03-13 15:58:00.958 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016771, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:00.966 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,c8b612cba9c7] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016770, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014417, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:00.943+08:00, sub_task_instance_name=Collect SlowSql to SqlAggregateData for alarm, sub_task_instance_id=10016770}, listMap={}}, executor=10.10.10.148 2024-03-13 15:58:00.967 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016772, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:00.968 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,ca932cb408e4] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:58:00.968 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,67cb64ed95a3] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:58:00.968 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,1cb788e587c8] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:58:00.968 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,eafcc43bed48] o.i.t.TenantCompactionCollectServiceImpl : Begin to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:58:00.973 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,e34a89ce2e19] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016771, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014418, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:00.955+08:00, sub_task_instance_name=Collect trans stat to SqlAggregateData for alarm, sub_task_instance_id=10016771}, listMap={}}, executor=10.10.10.148 2024-03-13 15:58:00.975 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,67cb64ed95a3] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:00.976 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,1cb788e587c8] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:00.976 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,67cb64ed95a3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:00.977 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,1cb788e587c8] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:00.977 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,1cb788e587c8] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:00.977 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,67cb64ed95a3] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:00.977 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,67cb64ed95a3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:00.977 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,1cb788e587c8] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:58:00.978 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,67cb64ed95a3] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:58:00.980 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,ca932cb408e4] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:00.980 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,ca932cb408e4] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:00.980 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,ca932cb408e4] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:00.981 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,1cb788e587c8] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266400687471944, broadcast scn:1710266400687471944, saved frozen scn:1710266400687471944 2024-03-13 15:58:00.981 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,67cb64ed95a3] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402011840869, broadcast scn:1710266402011840869, saved frozen scn:1710266402011840869 2024-03-13 15:58:00.981 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,eafcc43bed48] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:00.981 INFO 1433467 --- [pool-Collect all tenants major compactions5,f47a761474a84dce,1cb788e587c8] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:ocp 2024-03-13 15:58:00.981 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,ca932cb408e4] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1] 2024-03-13 15:58:00.981 INFO 1433467 --- [pool-Collect all tenants major compactions7,f47a761474a84dce,67cb64ed95a3] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster, tenant:sys 2024-03-13 15:58:00.981 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,eafcc43bed48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:00.981 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,eafcc43bed48] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:00.981 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,eafcc43bed48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:00.982 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,eafcc43bed48] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT tenant_id, global_broadcast_scn AS broadcast_scn, is_error AS error, status, frozen_scn, last_scn, is_suspended AS suspend, info, start_time, last_finish_time FROM CDB_OB_MAJOR_COMPACTION WHERE tenant_id = ?, args: [1002] 2024-03-13 15:58:00.983 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,ca932cb408e4] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266403004770377, broadcast scn:1710266403004770377, saved frozen scn:1710266403004770377 2024-03-13 15:58:00.983 INFO 1433467 --- [pool-Collect all tenants major compactions4,f47a761474a84dce,ca932cb408e4] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:sys 2024-03-13 15:58:00.984 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,eafcc43bed48] o.i.t.TenantCompactionCollectServiceImpl : current compaction frozen scn:1710266402269282986, broadcast scn:1710266402269282986, saved frozen scn:1710266402269282986 2024-03-13 15:58:00.984 INFO 1433467 --- [pool-Collect all tenants major compactions2,f47a761474a84dce,eafcc43bed48] o.i.t.TenantCompactionCollectServiceImpl : Finish to collect major compaction info for cluster:obcluster1, tenant:tenant1 2024-03-13 15:58:00.984 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,16db867fedfa] o.s.t.b.t.CollectAllTenantCompactionTask : Finish to collect all tenants major compaction info 2024-03-13 15:58:00.985 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016773, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:00.985 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,83bdbc5e4ab8] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016772, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014419, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:00.965+08:00, sub_task_instance_name=Get all agent alarms, sub_task_instance_id=10016772}, listMap={}}, executor=10.10.10.148 2024-03-13 15:58:00.988 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,16db867fedfa] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016769, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:00.995 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016774, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:00.997 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,e34a89ce2e19] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster1, trans stat result is empty. 2024-03-13 15:58:00.997 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,e34a89ce2e19] c.o.o.p.t.i.alarm.TransStatDataSource : query trans stat of obCluster obcluster, trans stat result is empty. 2024-03-13 15:58:00.997 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,4d88bde9b058] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016773, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014420, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:00.982+08:00, sub_task_instance_name=Refresh root service list, sub_task_instance_id=10016773}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:58:01.002 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,e34a89ce2e19] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016771, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:01.005 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016775, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:01.012 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,c8b612cba9c7] c.o.o.p.s.i.alarm.SlowSqlDataSource : collect SlowSqlData for obcluster1, save 1 to SqlAggregateData 2024-03-13 15:58:01.014 INFO 1433467 --- [pool-subtask-executor32,f47a761474a84dce,4d88bde9b058] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016773, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:01.015 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,c8b612cba9c7] c.o.o.p.s.i.alarm.SlowSqlDataSource : query sqlCount of obCluster obcluster, sql count is 0. 2024-03-13 15:58:01.015 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,c8b612cba9c7] c.o.o.p.s.i.alarm.SlowSqlDataSource : query SlowSqlData of obCluster obcluster, result is empty. 2024-03-13 15:58:01.017 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,b7adcaffad79] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016774, context=Context{parallelIdx=1, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014420, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:00.994+08:00, sub_task_instance_name=Refresh root service list, sub_task_instance_id=10016774}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:58:01.017 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,f63525a559d2] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016775, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014421, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:01.002+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016775}, listMap={}}, executor=10.10.10.148 2024-03-13 15:58:01.021 INFO 1433467 --- [pool-subtask-executor7,f47a761474a84dce,c8b612cba9c7] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016770, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:01.025 INFO 1433467 --- [pool-Check obproxy7,f47a761474a84dce,b9995ee01348] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:58:01.025 INFO 1433467 --- [pool-Check obproxy6,f47a761474a84dce,421e4a065880] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:58:01.028 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,66a09cca8006] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:58:01.029 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,b7adcaffad79] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016774, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:01.030 INFO 1433467 --- [pool-Check obproxy9,f47a761474a84dce,f7ceedb4e29f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:58:01.041 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=5 2024-03-13 15:58:01.041 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=4 2024-03-13 15:58:01.041 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=7 2024-03-13 15:58:01.041 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=3 2024-03-13 15:58:01.044 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:58:01.044 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:58:01.045 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:58:01.045 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:58:01.046 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:58:01.047 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:58:01.047 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:58:01.048 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:58:01.049 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,83bdbc5e4ab8] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016772, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:01.049 INFO 1433467 --- [pool-Check obproxy6,f47a761474a84dce,421e4a065880] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:58:01.055 INFO 1433467 --- [pool-Check obproxy1,f47a761474a84dce,66a09cca8006] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:58:01.056 INFO 1433467 --- [ForkJoinPool-15-worker-0,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62889/api/v1/log/alarms, params:null 2024-03-13 15:58:01.057 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=1 2024-03-13 15:58:01.057 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=6 2024-03-13 15:58:01.058 INFO 1433467 --- [pool-Check obproxy7,f47a761474a84dce,b9995ee01348] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:58:01.059 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Get OCP agent alarms: hostId=2 2024-03-13 15:58:01.060 INFO 1433467 --- [ForkJoinPool-15-worker-6,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62889/api/v1/log/alarms, params:null 2024-03-13 15:58:01.062 INFO 1433467 --- [ForkJoinPool-15-worker-3,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62889/api/v1/log/alarms, params:null 2024-03-13 15:58:01.062 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:58:01.062 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:58:01.063 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:58:01.063 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:58:01.063 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:58:01.064 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:58:01.066 INFO 1433467 --- [pool-Check obproxy9,f47a761474a84dce,f7ceedb4e29f] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:58:01.069 INFO 1433467 --- [ForkJoinPool-15-worker-4,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62889/api/v1/log/alarms, params:null 2024-03-13 15:58:01.074 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,f63525a559d2] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016775, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:01.074 INFO 1433467 --- [ForkJoinPool-15-worker-2,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62889/api/v1/log/alarms, params:null 2024-03-13 15:58:01.075 INFO 1433467 --- [ForkJoinPool-15-worker-7,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62889/api/v1/log/alarms, params:null 2024-03-13 15:58:01.081 INFO 1433467 --- [ForkJoinPool-15-worker-5,,] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62889/api/v1/log/alarms, params:null 2024-03-13 15:58:01.084 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:57:01.0, nextRun=2024-03-13 15:58:00.0, currentDate=Wed Mar 13 15:58:01 CST 2024 2024-03-13 15:58:01.084 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=38, name=Detect alarms on schedule, templateName=BackupScheduleAlarmTemplateFactory, scheduleRule=0 */1 * * * *, lastRunTime=2024-03-13 15:57:01.0, nextRunTime=2024-03-13 15:58:00.0} 2024-03-13 15:58:01.086 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Detect alarms, nodeCount=1, taskCount=1 2024-03-13 15:58:01.086 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:01.086 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Detect alarms, taskDefinitionId=38, state=RUNNING, operation=EXECUTE} 2024-03-13 15:58:01.093 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:56:01.0, nextRun=2024-03-13 15:58:00.0, currentDate=Wed Mar 13 15:58:01 CST 2024 2024-03-13 15:58:01.093 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999975, name=OasSqlCuring, templateName=OasSqlCuringSchedule, scheduleRule=0 */2 * * * *, lastRunTime=2024-03-13 15:56:01.0, nextRunTime=2024-03-13 15:58:00.0} 2024-03-13 15:58:01.107 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : Split list is empty, name=OasSqlCuringSchedule, splitKey=cluster_ids, context=Argument{parallelIdx=-1, stringMap={}, listMap={cluster_ids=[]}} 2024-03-13 15:58:01.107 WARN 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule cron task failed. java.lang.RuntimeException: Illegal split list, the list is empty, splitKey=cluster_ids at com.oceanbase.ocp.core.metadb.repo.task.model.TaskInstanceEntity.generateSubtasks(TaskInstanceEntity.java:150) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.buildTaskInstance(ReadyTaskWorker.java:270) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.submitTask(ReadyTaskWorker.java:210) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.doScheduleTask(ReadyTaskWorker.java:197) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.checkAndScheduleCronTask(ReadyTaskWorker.java:173) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.lambda$loopScheduleTask$1(ReadyTaskWorker.java:133) at org.springframework.transaction.support.TransactionOperations.lambda$executeWithoutResult$0(TransactionOperations.java:68) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) at org.springframework.transaction.support.TransactionOperations.executeWithoutResult(TransactionOperations.java:67) at com.oceanbase.ocp.core.task.engine.coordinator.worker.AbstractTaskWorker.transactionWithoutResult(AbstractTaskWorker.java:62) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.loopScheduleTask(ReadyTaskWorker.java:132) at com.oceanbase.ocp.core.task.engine.coordinator.worker.ReadyTaskWorker.work(ReadyTaskWorker.java:101) at com.oceanbase.ocp.core.task.engine.coordinator.TaskCoordinator.lockAndExecute(TaskCoordinator.java:94) at com.oceanbase.ocp.core.task.engine.coordinator.TaskCoordinator.lambda$startup$0(TaskCoordinator.java:65) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:01.222 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:01.223 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:01.617 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014416, status:SUCCESS 2024-03-13 15:58:01.618 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014416, status:SUCCESS 2024-03-13 15:58:01.626 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014417, status:SUCCESS 2024-03-13 15:58:01.627 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014417, status:SUCCESS 2024-03-13 15:58:01.636 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014418, status:SUCCESS 2024-03-13 15:58:01.636 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014418, status:SUCCESS 2024-03-13 15:58:01.645 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014419, status:SUCCESS 2024-03-13 15:58:01.645 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014419, status:SUCCESS 2024-03-13 15:58:01.653 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014420, status:SUCCESS 2024-03-13 15:58:01.654 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014420, status:SUCCESS 2024-03-13 15:58:01.661 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014421, status:SUCCESS 2024-03-13 15:58:01.661 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014421, status:SUCCESS 2024-03-13 15:58:01.929 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806813,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:01.994 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,c90b72e97ff64d15,449fbd3c4d36] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.142, traceId=c90b72e97ff64d15, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:58:02.001 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-190,c90b72e97ff64d15,449fbd3c4d36] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.142, traceId=c90b72e97ff64d15, duration=7 ms] 2024-03-13 15:58:02.024 INFO 1433467 --- [pool-subtask-coordinator2,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016776, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:02.045 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016776, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014422, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:02.022+08:00, sub_task_instance_name=Detect alarms of backup, sub_task_instance_id=10016776}, listMap={}}, executor=10.10.10.148 2024-03-13 15:58:02.070 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.073 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.076 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.080 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:58:02.139 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:02.142 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.145 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.148 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.150 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:58:02.167 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:58:02.185 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.189 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.191 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.194 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'FAILED' AND end_time > date_sub(now(), INTERVAL ? DAY), args: [2] 2024-03-13 15:58:02.234 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:02.236 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.239 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.241 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.243 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid RIGHT JOIN (SELECT max(job_id) as _job_id, tenant_id as _tenant_id FROM ( SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, tenant_id, backup_type, status FROM CDB_OB_BACKUP_JOB_HISTORY ) WHERE tenant_id != 1 AND backup_type = 'FULL' AND status = 'COMPLETED' GROUP BY tenant_id ) AS t ON job_id = t._job_id AND tenant_id = t._tenant_id ORDER BY start_time DESC 2024-03-13 15:58:02.254 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : maxAlarmDays=2, failedTasks=[] 2024-03-13 15:58:02.270 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:02.272 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.285 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:02.287 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.289 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.292 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.296 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:58:02.300 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:58:02.303 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.305 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.309 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [3600] 2024-03-13 15:58:02.346 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=3600, delayTasks=[] 2024-03-13 15:58:02.361 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:02.365 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.374 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:02.377 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.379 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.381 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.383 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW VARIABLES LIKE 'time_zone' 2024-03-13 15:58:02.385 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.operation.BackupObOpsService : systemTenantZoneId=+08:00 2024-03-13 15:58:02.388 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.391 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.395 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT incarnation, round_id AS log_archive_round, tenant_id, path AS backup_dest, IF(start_scn_display != '', start_scn_display, NULL) AS min_first_time, IF(checkpoint_scn_display != '', checkpoint_scn_display, NULL) AS max_next_time, status, IF(checkpoint_scn != '', truncate((time_to_usec(now()) - checkpoint_scn / 1000) / 1000000, 4), NULL) AS delay, now(6) AS check_time FROM CDB_OB_ARCHIVELOG_SUMMARY RIGHT JOIN (SELECT tenant_id as _tenant_id, max(round_id) as _round_id FROM CDB_OB_ARCHIVELOG_SUMMARY GROUP BY _tenant_id ) AS t ON tenant_id = t._tenant_id and round_id = t._round_id) WHERE delay >= ?, args: [300] 2024-03-13 15:58:02.427 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : minDelaySeconds=300, delayTasks=[] 2024-03-13 15:58:02.446 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:02.450 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.454 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.457 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.461 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [90] 2024-03-13 15:58:02.497 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=90, timeoutTasks=[] 2024-03-13 15:58:02.507 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:02.510 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.513 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.516 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.519 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE (status = 'DOING' OR status = 'INIT') AND start_time < date_sub(now(), INTERVAL ? MINUTE), args: [60] 2024-03-13 15:58:02.553 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : minTimeoutMinutes=60, timeoutTasks=[] 2024-03-13 15:58:02.572 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:02.575 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.578 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:02.580 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.585 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:58:02.639 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:58:02.656 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:02.661 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.670 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:02.673 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:02.680 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED' AND end_time < date_sub(now(), INTERVAL ? DAY) AND end_time IN ( SELECT max(end_time) FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1) WHERE status = 'COMPLETED'), args: [2] 2024-03-13 15:58:02.711 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : minAlarmDays=2, tooOldTasks=[] 2024-03-13 15:58:02.714 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster1 does not enable secondary backup 2024-03-13 15:58:02.717 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : cluster=obcluster does not enable secondary backup 2024-03-13 15:58:02.727 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackup/], metricValues size:[0]. 2024-03-13 15:58:02.732 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.b.i.alarm.BackupAlarmService : search root dir:[file:///data/nfs/obbackupocp/], metricValues size:[0]. 2024-03-13 15:58:02.744 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,94b71987fbdb] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016776, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:03.930 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806836,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:04.165 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014422, status:SUCCESS 2024-03-13 15:58:04.166 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014422, status:SUCCESS 2024-03-13 15:58:04.223 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:04.224 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:05.930 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806859,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:06.112 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806862,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:58:06.746 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:58:06.746295+08:00, lastProcessTime=2024-03-13T15:57:56.747087+08:00 2024-03-13 15:58:06.843 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:58:06.746295+08:00 2024-03-13 15:58:07.041 INFO 1433467 --- [pool-ocp-schedules-14,dee32a2b9f7c4847,6853795dcfc9] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:58:07.225 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:07.226 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:07.931 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806883,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:09.249 INFO 1433467 --- [pool-ocp-schedules-16,ad190a6b5bae4cf1,c5f46db2ca83] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] begin refresh method to audit event meta map... 2024-03-13 15:58:09.256 INFO 1433467 --- [pool-ocp-schedules-13,b67b472cfb6b4251,45de7023c268] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:58:09.256 INFO 1433467 --- [pool-ocp-schedules-20,2246f2cd10fd4687,cb61ae7991bd] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:58:09.257 INFO 1433467 --- [pool-ocp-schedules-17,4d7ec7692edc42ec,ef6fc9d8f110] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:58:09.260 INFO 1433467 --- [pool-ocp-schedules-20,2246f2cd10fd4687,cb61ae7991bd] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:58:09.260 INFO 1433467 --- [pool-ocp-schedules-13,b67b472cfb6b4251,45de7023c268] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:58:09.260 INFO 1433467 --- [pool-ocp-schedules-17,4d7ec7692edc42ec,ef6fc9d8f110] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:58:09.262 INFO 1433467 --- [pool-ocp-schedules-16,ad190a6b5bae4cf1,c5f46db2ca83] c.o.ocp.service.audit.AuditEventService : [refreshMethodToAuditEventMetaIdMap] refresh method to audit event meta map success 2024-03-13 15:58:09.268 INFO 1433467 --- [pool-ocp-schedules-10,7b52af504d884de5,91e689af1beb] c.o.o.s.task.scheduled.ClusterSchedules : Check host version schedule. 2024-03-13 15:58:09.268 INFO 1433467 --- [pool-ocp-schedules-10,7b52af504d884de5,91e689af1beb] o.o.c.t.e.u.DistributedExclusiveExecutor : Empty item list, taskName=Check host task for host version, skip execute 2024-03-13 15:58:09.269 INFO 1433467 --- [pool-ocp-schedules-4,d95be4bbb0dc4809,c6b58bfc0a55] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:58:09.270 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:58:09.270 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:58:09.270 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:58:09.270 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:58:09.270 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:58:09.270 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:58:09.270 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:58:09.274 INFO 1433467 --- [pool-ocp-schedules-3,6164364ce08c4390,84fc350be4b4] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:58:09.274 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:58:09.275 INFO 1433467 --- [pool-ocp-schedules-6,7a1dfae6eeec4390,aee2ad14c087] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:58:09.275 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:58:09.275 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:58:09.275 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:58:09.276 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:58:09.276 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:58:09.276 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:58:09.277 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:58:09.277 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:58:09.278 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:58:09.278 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:58:09.278 INFO 1433467 --- [pool-ocp-schedules-19,25525d0fa156419f,80ff272945ac] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:58:09.279 INFO 1433467 --- [pool-Check host task for clockdiff1,25525d0fa156419f,98ef590c8785] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:58:09.279 INFO 1433467 --- [pool-Check host task for clockdiff3,25525d0fa156419f,36e903a7257a] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:58:09.279 INFO 1433467 --- [pool-Check host task for clockdiff21,25525d0fa156419f,6a7c115eb920] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:58:09.279 INFO 1433467 --- [pool-Check host task for clockdiff7,25525d0fa156419f,f8473154f313] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:58:09.279 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:58:09.279 INFO 1433467 --- [pool-Check host task for clockdiff27,25525d0fa156419f,31c914316a58] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:58:09.279 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:58:09.279 INFO 1433467 --- [pool-Check host task for clockdiff4,25525d0fa156419f,5ce2e4dbab56] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:58:09.279 INFO 1433467 --- [pool-Check host task for clockdiff28,25525d0fa156419f,eb362c7fb03f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:58:09.280 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:58:09.281 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806898,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:58:09.286 INFO 1433467 --- [pool-ocp-schedules-18,6b8e5f15d01b45eb,bd430b7dea05] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:58:09.286 INFO 1433467 --- [pool-Check host task for agent23,6b8e5f15d01b45eb,46820cb0ed49] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:58:09.286 INFO 1433467 --- [pool-Check host task for agent8,6b8e5f15d01b45eb,d8c7150a8ae4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:58:09.286 INFO 1433467 --- [pool-Check host task for agent9,6b8e5f15d01b45eb,a0b78240fdf1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:58:09.286 INFO 1433467 --- [pool-Check host task for agent7,6b8e5f15d01b45eb,a1230185228e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:58:09.286 INFO 1433467 --- [pool-Check host task for agent3,6b8e5f15d01b45eb,c4fe4e4a9a60] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:58:09.286 INFO 1433467 --- [pool-Check host task for agent1,6b8e5f15d01b45eb,0f7b464618c9] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:58:09.286 INFO 1433467 --- [pool-Check host task for agent5,6b8e5f15d01b45eb,93cbb0304019] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:58:09.291 INFO 1433467 --- [pool-Check host task for agent7,6b8e5f15d01b45eb,a1230185228e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:58:09.292 INFO 1433467 --- [pool-Check host task for agent5,6b8e5f15d01b45eb,93cbb0304019] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:58:09.292 INFO 1433467 --- [pool-Check host task for agent7,6b8e5f15d01b45eb,a1230185228e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:58:09.293 INFO 1433467 --- [pool-Check host task for agent23,6b8e5f15d01b45eb,46820cb0ed49] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:58:09.293 INFO 1433467 --- [pool-Check host task for agent5,6b8e5f15d01b45eb,93cbb0304019] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:58:09.294 INFO 1433467 --- [pool-Check host task for agent23,6b8e5f15d01b45eb,46820cb0ed49] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:58:09.294 INFO 1433467 --- [pool-Check host task for agent1,6b8e5f15d01b45eb,0f7b464618c9] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:58:09.294 INFO 1433467 --- [pool-Check host task for agent8,6b8e5f15d01b45eb,d8c7150a8ae4] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:58:09.296 INFO 1433467 --- [pool-Check host task for agent8,6b8e5f15d01b45eb,d8c7150a8ae4] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:58:09.296 INFO 1433467 --- [pool-Check host task for agent1,6b8e5f15d01b45eb,0f7b464618c9] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:58:09.297 INFO 1433467 --- [pool-Check host task for agent9,6b8e5f15d01b45eb,a0b78240fdf1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:58:09.298 INFO 1433467 --- [pool-Check host task for agent9,6b8e5f15d01b45eb,a0b78240fdf1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:58:09.305 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:58:09.305 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.305 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.305 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.305 INFO 1433467 --- [pool-Check host task for agent3,6b8e5f15d01b45eb,c4fe4e4a9a60] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:58:09.306 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:58:09.306 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:58:09.306 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:58:09.307 INFO 1433467 --- [pool-Check host task for agent3,6b8e5f15d01b45eb,c4fe4e4a9a60] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:58:09.307 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:58:09.307322956+08:00 2024-03-13 15:58:09.308 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:58:09.308 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:58:09.308 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:58:09.308326506+08:00 2024-03-13 15:58:09.308 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:58:09.308576170+08:00 2024-03-13 15:58:09.309 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:58:09.307+08:00 2024-03-13 15:58:09.309 INFO 1433467 --- [ForkJoinPool-1386597-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.309 INFO 1433467 --- [ForkJoinPool-1386597-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.309 INFO 1433467 --- [ForkJoinPool-1386597-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.310 INFO 1433467 --- [ForkJoinPool-1386597-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:58:09.310 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:58:09.310054237+08:00 2024-03-13 15:58:09.310 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:58:09.310118462+08:00 2024-03-13 15:58:09.311 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:58:09.310+08:00 2024-03-13 15:58:09.311 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:58:09.309+08:00 2024-03-13 15:58:09.311 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:58:09.310+08:00 2024-03-13 15:58:09.312 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:58:09.307+08:00 and status AVAILABLE 2024-03-13 15:58:09.312 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:58:09.308+08:00 2024-03-13 15:58:09.313 INFO 1433467 --- [ForkJoinPool-1386597-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:58:09.313 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:58:09.310+08:00 and status AVAILABLE 2024-03-13 15:58:09.314 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:58:09.309+08:00 and status AVAILABLE 2024-03-13 15:58:09.314 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:58:09.310+08:00 and status AVAILABLE 2024-03-13 15:58:09.316 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:58:09.308+08:00 and status AVAILABLE 2024-03-13 15:58:09.318 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:58:09.318 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:58:09.318 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:58:09.319 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:58:09.320 INFO 1433467 --- [pool-Check host task for agent7,6b8e5f15d01b45eb,a1230185228e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:58:09.321 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:58:09.322 INFO 1433467 --- [ForkJoinPool-1386597-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.322 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:58:09.322630795+08:00 2024-03-13 15:58:09.322 INFO 1433467 --- [ForkJoinPool-1386597-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.323 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:58:09.322+08:00 2024-03-13 15:58:09.323 INFO 1433467 --- [pool-Check host task for agent9,6b8e5f15d01b45eb,a0b78240fdf1] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:58:09.324 INFO 1433467 --- [pool-Check host task for agent7,6b8e5f15d01b45eb,a1230185228e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:58:09.322 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:58:09.324 INFO 1433467 --- [ForkJoinPool-1386597-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.324 INFO 1433467 --- [pool-Check host task for agent7,6b8e5f15d01b45eb,a1230185228e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:58:09.324 INFO 1433467 --- [pool-Check host task for agent7,6b8e5f15d01b45eb,a1230185228e] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:58:09.324 INFO 1433467 --- [ForkJoinPool-1386597-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:58:09.325 INFO 1433467 --- [ForkJoinPool-1386597-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:58:09.325 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:58:09.325 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:58:09.322+08:00 and status AVAILABLE 2024-03-13 15:58:09.326 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:58:09.326 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:58:09.328 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:58:09.327806779+08:00 2024-03-13 15:58:09.328 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:58:09.328 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:58:09.328+08:00 2024-03-13 15:58:09.329 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:58:09.329 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.329 INFO 1433467 --- [pool-Check host task for agent9,6b8e5f15d01b45eb,a0b78240fdf1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:58:09.329 INFO 1433467 --- [pool-Check host task for agent9,6b8e5f15d01b45eb,a0b78240fdf1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:58:09.329 INFO 1433467 --- [pool-Check host task for agent9,6b8e5f15d01b45eb,a0b78240fdf1] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:58:09.329 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.329 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.329 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:58:09.330 INFO 1433467 --- [pool-Check host task for agent1,6b8e5f15d01b45eb,0f7b464618c9] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:58:09.330 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:58:09.330 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:58:09.328+08:00 and status AVAILABLE 2024-03-13 15:58:09.330 INFO 1433467 --- [pool-Check host task for agent5,6b8e5f15d01b45eb,93cbb0304019] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:58:09.331 INFO 1433467 --- [pool-Check host task for agent8,6b8e5f15d01b45eb,d8c7150a8ae4] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:58:09.331 INFO 1433467 --- [pool-Check host task for agent23,6b8e5f15d01b45eb,46820cb0ed49] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:58:09.332 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:58:09.332 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:58:09.333 INFO 1433467 --- [pool-Check host task for agent8,6b8e5f15d01b45eb,d8c7150a8ae4] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:58:09.333 INFO 1433467 --- [pool-Check host task for agent8,6b8e5f15d01b45eb,d8c7150a8ae4] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:58:09.333 INFO 1433467 --- [pool-Check host task for agent8,6b8e5f15d01b45eb,d8c7150a8ae4] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:58:09.333 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:58:09.334 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:58:09.334 INFO 1433467 --- [pool-Check host task for agent23,6b8e5f15d01b45eb,46820cb0ed49] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:58:09.334 INFO 1433467 --- [pool-Check host task for agent23,6b8e5f15d01b45eb,46820cb0ed49] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:58:09.334 INFO 1433467 --- [pool-Check host task for agent23,6b8e5f15d01b45eb,46820cb0ed49] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:58:09.335 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:58:09.335 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:58:09.335 INFO 1433467 --- [pool-Check host task for agent1,6b8e5f15d01b45eb,0f7b464618c9] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:58:09.335 INFO 1433467 --- [pool-Check host task for agent1,6b8e5f15d01b45eb,0f7b464618c9] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:58:09.335 INFO 1433467 --- [pool-Check host task for agent1,6b8e5f15d01b45eb,0f7b464618c9] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:58:09.335 INFO 1433467 --- [pool-Check host task for agent3,6b8e5f15d01b45eb,c4fe4e4a9a60] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:58:09.336 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:58:09.336 INFO 1433467 --- [pool-Check host task for agent5,6b8e5f15d01b45eb,93cbb0304019] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:58:09.336 INFO 1433467 --- [pool-Check host task for agent5,6b8e5f15d01b45eb,93cbb0304019] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:58:09.336 INFO 1433467 --- [pool-Check host task for agent5,6b8e5f15d01b45eb,93cbb0304019] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:58:09.336 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:58:09.338 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:58:09.340 INFO 1433467 --- [pool-Check host task for agent3,6b8e5f15d01b45eb,c4fe4e4a9a60] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:58:09.340 INFO 1433467 --- [pool-Check host task for agent3,6b8e5f15d01b45eb,c4fe4e4a9a60] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:58:09.340 INFO 1433467 --- [pool-Check host task for agent3,6b8e5f15d01b45eb,c4fe4e4a9a60] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:58:09.342 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:58:09.343 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:58:09.347 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:09.349 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:09.349 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:09.349 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:09.353 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:09.356 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:09.358 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:09.364 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:58:09.373 WARN 1433467 --- [pool-Check host task for clockdiff3,25525d0fa156419f,36e903a7257a] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.373 ERROR 1433467 --- [pool-Check host task for clockdiff3,25525d0fa156419f,36e903a7257a] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:58:09.373 WARN 1433467 --- [pool-Check host task for clockdiff3,25525d0fa156419f,36e903a7257a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.388 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:58:09.389 WARN 1433467 --- [pool-Check host task for clockdiff21,25525d0fa156419f,6a7c115eb920] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.389 ERROR 1433467 --- [pool-Check host task for clockdiff21,25525d0fa156419f,6a7c115eb920] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:58:09.389 WARN 1433467 --- [pool-Check host task for clockdiff21,25525d0fa156419f,6a7c115eb920] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.393 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:58:09.394 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:58:09.397 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:58:09.397 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:58:09.401 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:58:09.407 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:58:09.408 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:09.409 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:09.414 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:09.418 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:58:09.419 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:58:09.419 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:58:09.419 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:09.430 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:58:09.433 WARN 1433467 --- [pool-Check host task for clockdiff28,25525d0fa156419f,eb362c7fb03f] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.433 ERROR 1433467 --- [pool-Check host task for clockdiff28,25525d0fa156419f,eb362c7fb03f] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:58:09.435 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:58:09.446 WARN 1433467 --- [pool-scanner-1609807505-1,5576c41829374e88,c269186269b0] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:58:09.451 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:58:09.451 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:58:09.457 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:58:09.457 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:58:09.465 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:09.466 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:09.467 WARN 1433467 --- [pool-Check host task for clockdiff4,25525d0fa156419f,5ce2e4dbab56] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.467 ERROR 1433467 --- [pool-Check host task for clockdiff4,25525d0fa156419f,5ce2e4dbab56] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:58:09.467 WARN 1433467 --- [pool-Check host task for clockdiff4,25525d0fa156419f,5ce2e4dbab56] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.470 WARN 1433467 --- [pool-Check host task for clockdiff27,25525d0fa156419f,31c914316a58] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.470 ERROR 1433467 --- [pool-Check host task for clockdiff27,25525d0fa156419f,31c914316a58] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:58:09.470 WARN 1433467 --- [pool-Check host task for clockdiff27,25525d0fa156419f,31c914316a58] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.471 WARN 1433467 --- [pool-Check host task for clockdiff28,25525d0fa156419f,eb362c7fb03f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.471 WARN 1433467 --- [pool-Check host task for clockdiff1,25525d0fa156419f,98ef590c8785] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.471 ERROR 1433467 --- [pool-Check host task for clockdiff1,25525d0fa156419f,98ef590c8785] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:58:09.471 WARN 1433467 --- [pool-Check host task for clockdiff7,25525d0fa156419f,f8473154f313] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.471 WARN 1433467 --- [pool-Check host task for clockdiff1,25525d0fa156419f,98ef590c8785] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.471 ERROR 1433467 --- [pool-Check host task for clockdiff7,25525d0fa156419f,f8473154f313] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:58:09.471 WARN 1433467 --- [pool-Check host task for clockdiff7,25525d0fa156419f,f8473154f313] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:09.471 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:58:09.471 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:58:09.471 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:58:09.479 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:58:09.481 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:09.483 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:09.486 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:09.487 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:58:09.489 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:09.490 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.490 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:09.490 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.490 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.491 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:58:09.495 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:09.502 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:58:09.505 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:58:09.505 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:58:09.507 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:58:09.509 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:58:09.512 INFO 1433467 --- [pool-Check host task for host45,d95be4bbb0dc4809,24c32fe416e7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:58:09.513 INFO 1433467 --- [pool-Check host task for host1,d95be4bbb0dc4809,28e6df407f04] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:58:09.513 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:09.514 INFO 1433467 --- [pool-Check host task for host24,d95be4bbb0dc4809,ab3216c5dd27] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:58:09.514 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.514 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.514 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.515 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:58:09.515 WARN 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:58:09.516 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.516 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.517 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:58:09.517 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:58:09.517 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:58:09.518 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:58:09.523 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:09.523 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:58:09.523 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:58:09.529 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:09.529 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:09.529 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:58:09.529 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:09.535 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:09.535 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.535 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.535 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.536 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:58:09.536 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:58:09.538 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:58:09.540 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:58:09.542 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:58:09.546 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:09.547 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:58:09.553 INFO 1433467 --- [pool-Check host task for host49,d95be4bbb0dc4809,d8bad77935f0] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:58:09.553 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:58:09.554 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:58:09.557 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:58:09.560 INFO 1433467 --- [pool-Check host task for host52,d95be4bbb0dc4809,25f453795140] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:58:09.565 INFO 1433467 --- [pool-Check host task for host50,d95be4bbb0dc4809,86593d5b0477] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:58:09.594 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:58:09.605 INFO 1433467 --- [pool-Check host task for host56,d95be4bbb0dc4809,7f3534157332] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:58:09.614 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:58:09.620 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:09.621 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.621 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:09.621 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.622 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:58:09.627 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:58:09.627 INFO 1433467 --- [ForkJoinPool-1386599-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316540069000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:58:09.637 INFO 1433467 --- [alarm-detect-25,e41928ae71dd4812,08a1e01823c7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000913, data is empty. 2024-03-13 15:58:09.640 INFO 1433467 --- [alarm-detect-25,e41928ae71dd4812,08a1e01823c7] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000914, data is empty. 2024-03-13 15:58:09.681 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:09.681 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.682 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.682 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.682 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:58:09.704 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:09.704 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.704 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.704 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.705 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:58:09.707 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.707 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.707 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:58:09.710 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:58:09.720 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:09.720 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:58:09.729 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:09.729 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:58:09.736 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:09.736 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:58:09.744 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:09.744 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:58:09.751 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:09.751 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:58:09.758 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:09.771 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:09.771 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.772 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.772 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.772 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:58:09.776 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:58:09.780 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:58:09.784 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:58:09.786 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:58:09.810 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:58:09.844 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:58:09.867 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:09.867 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.868 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:09.868 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:09.868 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:58:09.882 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:58:09.882 INFO 1433467 --- [ForkJoinPool-1386599-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:58:09.931 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806908,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:10.226 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:10.227 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:10.392 INFO 1433467 --- [pool-async-queue-offer-interval60-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 60, queue size = 6, ruleSize = 6. 2024-03-13 15:58:10.400 INFO 1433467 --- [pool-async-queue-offer-interval5-1,,] c.o.o.m.a.MonitorAsyncComputeProcessor : Offer async queue, interval = 5, queue size = 17, ruleSize = 17. 2024-03-13 15:58:10.978 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:58:10.979 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:58:10.983 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:58:10.983 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:58:10.988 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:58:10.988 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:58:10.993 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:58:10.993 WARN 1433467 --- [pool-async-compute-interval60-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:58:11.393 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:58:11.394 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:58:11.420 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:58:11.421 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:58:11.422 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:58:11.423 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:58:11.445 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:58:11.446 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:58:11.466 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = major_freeze_trigger 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_size 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_hit_ratio 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_logonly_clog_sync_delay_seconds 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_fail_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_total_waits 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_memory_percent 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_req_total 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_total_waits 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_limit 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_req_total 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_total_waits 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_memstore_used 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_enqueue_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_full_clog_sync_delay_seconds 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_size 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_size 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_percent 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_wait_time 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_sync_rpc_time_waited 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_hit_ratio 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = table_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_time 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = opened_cursors_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_distributed_total_used_time 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_total_waits 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_cpu_percent 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_out_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_size 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_distributed_time 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memstore_write_lock_succ_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_req_total 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_total_waits 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_sp_total_used_time 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = plan_cache_hit_ratio 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_req_total 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_total_waits 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = block_cache_hit_ratio 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_latch_time_waited 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = rpc_packet_in_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_internal_time_waited 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = max_readonly_clog_sync_delay_seconds 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_time 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_size 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = wait_event_rt 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_other_time_waited 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_cache_hit_ratio 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_row_lock_wait_time_waited 2024-03-13 15:58:11.467 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_remote_count 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_local_count 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = row_cache_size 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = ob_tenant_thread_percent 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = total_memstore_used 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = location_cache_hit_ratio 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_dequeue_count 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = bloom_filter_cache_req_total 2024-03-13 15:58:11.468 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = system_event_io_time_waited 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:58:11.476 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_queue_time 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_single_partition_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_delete_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_size 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_timeout_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_sync_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_insert_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_size 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = all_session 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_rollback_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_select_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_size 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_update_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_read_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = active_session 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_commit_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_multi_partition_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_trans_log_total_size 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_other_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_write_disk_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_rt 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = tps 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = sql_replace_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = trans_commit_log_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_count 2024-03-13 15:58:11.485 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = clog_io_write_size 2024-03-13 15:58:11.486 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:58:11.486 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:58:11.486 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total 2024-03-13 15:58:11.486 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:58:11.489 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_route 2024-03-13 15:58:11.489 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_server_process_request_time 2024-03-13 15:58:11.489 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_hit 2024-03-13 15:58:11.489 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_insert 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_update 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_response 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_sequence 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_partition 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_table 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_request 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_server_request 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_client 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_byte_client_response 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_commit 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_delete 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_update 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_select 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_slow 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_others 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_sequence 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_large 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_prepare_send_request_time 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = entry_total_success 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = current_session_server 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_select 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = transaction_total 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_delete 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_commit 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_total_others 2024-03-13 15:58:11.490 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = request_cost_insert 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:58:11.492 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_free 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_cached 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_used 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = memory_buffers 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:58:11.495 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:58:11.497 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:58:11.499 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:58:11.499 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:58:11.499 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:58:11.499 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:58:11.499 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:58:11.499 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:58:11.499 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:58:11.499 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:58:11.499 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:58:11.500 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:58:11.500 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:58:11.500 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:58:11.500 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:58:11.500 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:58:11.500 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:58:11.500 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:58:11.500 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:58:11.502 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:58:11.504 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:58:11.507 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:58:11.509 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_qusize 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_iowait 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_user 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load5 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_system 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load1 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_recv 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_byte 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_util 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = net_send 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_time 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_read_byte 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = io_write_time 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = load15 2024-03-13 15:58:11.511 WARN 1433467 --- [pool-async-compute-interval5-1,,] c.o.o.m.s.MonitorAsyncComputeServiceImpl : queryMetricByMonitorRules get empty results. metric = cpu_percent 2024-03-13 15:58:11.932 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806931,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:12.453 INFO 1433467 --- [pool-metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316620, nodeCount=624, affectRows=624 2024-03-13 15:58:12.455 INFO 1433467 --- [pool-metric-archive-second-4,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316620, nodeCount=623, validCount=623 2024-03-13 15:58:12.458 INFO 1433467 --- [pool-metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316620, nodeCount=639, affectRows=639 2024-03-13 15:58:12.458 INFO 1433467 --- [pool-metric-archive-second-3,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316620, nodeCount=854, affectRows=854 2024-03-13 15:58:12.463 INFO 1433467 --- [pool-metric-archive-second-3,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316620, nodeCount=854, validCount=854 2024-03-13 15:58:12.465 INFO 1433467 --- [pool-metric-archive-second-6,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316620, nodeCount=639, validCount=639 2024-03-13 15:58:12.466 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Second type metric archive, timestamp=1710316620, nodeCount=899, affectRows=899 2024-03-13 15:58:12.469 INFO 1433467 --- [pool-metric-archive-second-1,,] c.o.o.m.service.OcpMetricArchiveService : Rollup second to minute, timestamp=1710316620, nodeCount=899, validCount=899 2024-03-13 15:58:13.227 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:13.228 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:13.257 INFO 1433467 --- [pool-ocp-schedules-7,f7d70c6d2bd843c9,bd62ca34d965] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:58:13.264 INFO 1433467 --- [pool-ocp-schedules-7,f7d70c6d2bd843c9,bd62ca34d965] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:58:13.264 INFO 1433467 --- [pool-ocp-schedules-7,f7d70c6d2bd843c9,bd62ca34d965] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:58:13.932 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806955,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:14.636 INFO 1433467 --- [pool-metric-archive-async-8,,] c.o.o.m.service.OcpMetricArchiveService : async type metric archive, timestamp=1710316620, nodeCount=2, affectRows=2 2024-03-13 15:58:14.643 INFO 1433467 --- [pool-metric-archive-async-8,,] c.o.o.m.service.OcpMetricArchiveService : Rollup async second to minute, timestamp=1710316620, nodeCount=2, validCount=2 2024-03-13 15:58:15.102 INFO 1433467 --- [pool-prometheus-second-schedule-1,16b4b9e2fdfa40b1,011da4778592] c.o.ocp.monitor.OcpMonitorManager : create pull exporter, exporterCount=23, collectTaskCount=23 2024-03-13 15:58:15.466 INFO 1433467 --- [pool-10-thread-10,2e5169d1b35d4c79,65703d259e5a] c.o.o.p.t.i.ObTransactionDiagServiceImpl : begin to get cluster trans stat 2024-03-13 15:58:15.466 INFO 1433467 --- [pool-10-thread-11,2e5169d1b35d4c79,1898e649d4a6] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get outline history 2024-03-13 15:58:15.467 INFO 1433467 --- [pool-10-thread-12,2e5169d1b35d4c79,ac5410213d84] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get Cluster Sql Stat 2024-03-13 15:58:15.467 INFO 1433467 --- [pool-10-thread-12,2e5169d1b35d4c79,ac5410213d84] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get cluster slow Sql Stat 2024-03-13 15:58:15.469 INFO 1433467 --- [pool-10-thread-9,2e5169d1b35d4c79,6669334f847d] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get cluster session stat 2024-03-13 15:58:15.471 INFO 1433467 --- [pool-10-thread-10,2e5169d1b35d4c79,65703d259e5a] c.o.o.p.t.i.ObTransactionDiagServiceImpl : begin to attach trans id 2024-03-13 15:58:15.471 INFO 1433467 --- [pool-10-thread-10,2e5169d1b35d4c79,65703d259e5a] c.o.o.p.t.i.ObTransactionDiagServiceImpl : finish fetching trans data 2024-03-13 15:58:15.472 INFO 1433467 --- [pool-10-thread-12,2e5169d1b35d4c79,ac5410213d84] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get cluster anomaly sql Stat 2024-03-13 15:58:15.473 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get tenant session stat of obcluster 2024-03-13 15:58:15.473 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.o.p.o.i.OasSessionStatServiceImpl : begin to get session of obcluster 2024-03-13 15:58:15.474 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.o.p.oas.internal.OasStatServiceImpl : begin to get tenant session stat of obcluster1 2024-03-13 15:58:15.474 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.o.p.o.i.OasSessionStatServiceImpl : begin to get session of obcluster1 2024-03-13 15:58:15.475 INFO 1433467 --- [pool-10-thread-12,2e5169d1b35d4c79,ac5410213d84] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching cluster sql stat 2024-03-13 15:58:15.490 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.491 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.491 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.491 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2024-03-13 15:58:15.492 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.492 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.492 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.492 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2024-03-13 15:58:15.493 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.493 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.493 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.494 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state ='ACTIVE' 2024-03-13 15:58:15.496 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.496 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.496 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2024-03-13 15:58:15.499 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.500 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.500 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.500 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state ='ACTIVE' 2024-03-13 15:58:15.502 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=100, clusterName=obcluster, tenantName=ocp, dbUser=root 2024-03-13 15:58:15.503 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.503 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.503 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.503 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=100, clusterName=obcluster1, tenantName=sys, dbUser=root 2024-03-13 15:58:15.503 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.503 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2024-03-13 15:58:15.503 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.504 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.504 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.504 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(60000000) */ time_to_usec(t1.MODIFY_TIME) AS UPDATE_TIME, t1.RESOURCE_POOL_ID, t1.NAME, t1.UNIT_COUNT, t1.UNIT_CONFIG_ID, t1.ZONE_LIST, t1.TENANT_ID, t1.REPLICA_TYPE, t2.NAME AS UNIT_CONFIG_NAME, t2.MAX_CPU, t2.MIN_CPU, t2.MEMORY_SIZE AS MAX_MEMORY, t2.MEMORY_SIZE AS MIN_MEMORY, t2.LOG_DISK_SIZE, t2.MAX_IOPS, t2.MIN_IOPS, t2.IOPS_WEIGHT FROM oceanbase.DBA_OB_RESOURCE_POOLS AS t1 JOIN oceanbase.DBA_OB_UNIT_CONFIGS AS t2 ON t1.UNIT_CONFIG_ID = t2.UNIT_CONFIG_ID AND t1.tenant_id = ?, args: [1002] 2024-03-13 15:58:15.504 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.504 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.504 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.504 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.504 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2024-03-13 15:58:15.505 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2024-03-13 15:58:15.507 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.507 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.507 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.508 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.508 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.508 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state ='ACTIVE' 2024-03-13 15:58:15.508 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2024-03-13 15:58:15.509 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.509 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.509 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.509 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.509 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2024-03-13 15:58:15.509 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2024-03-13 15:58:15.509 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.o.p.o.i.OasSessionStatServiceImpl : begin to get deadlock of obcluster 2024-03-13 15:58:15.515 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=100, clusterName=obcluster, tenantName=ocp, dbUser=root 2024-03-13 15:58:15.515 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.523 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.523 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.523 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY, args: [] 2024-03-13 15:58:15.525 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:15.525 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.527 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.528 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.528 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.528 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2024-03-13 15:58:15.533 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.533 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.533 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.533 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(5000000) */ t1.UNIT_ID, t1.RESOURCE_POOL_ID, t1.UNIT_CONFIG_ID, t1.ZONE, t1.SVR_IP, t1.SVR_PORT, t1.STATUS, t2.REPLICA_TYPE, t2.TENANT_ID, t3.TENANT_NAME, t2.NAME AS RESOURCE_POOL_NAME, t2.MODIFY_TIME AS RESOURCE_POOL_UPDATE_TIME, t1.MIGRATE_FROM_SVR_IP, t1.MIGRATE_FROM_SVR_PORT, t1.MANUAL_MIGRATE FROM oceanbase.DBA_OB_UNITS AS t1 JOIN oceanbase.DBA_OB_RESOURCE_POOLS AS t2 ON t1.RESOURCE_POOL_ID = t2.RESOURCE_POOL_ID JOIN oceanbase.DBA_OB_TENANTS t3 ON t2.TENANT_ID = t3.TENANT_ID WHERE 1=1 AND t2.tenant_id = ?, args: [1002] 2024-03-13 15:58:15.534 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.534 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.534 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.534 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? AND state ='ACTIVE' 2024-03-13 15:58:15.535 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.535 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.536 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, case enabled when 'YES' then 1 else 0 end as enabled, create_time as gmt_create from DBA_OB_OUTLINE_CONCURRENT_HISTORY where ADDDATE(create_time, ?) > CURDATE() and is_deleted = 'NO' 2024-03-13 15:58:15.537 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.537 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.538 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT COUNT(*) FROM GV$OB_PROCESSLIST WHERE 1=1 AND tenant = ? 2024-03-13 15:58:15.538 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=100, clusterName=obcluster, tenantName=sys, dbUser=root 2024-03-13 15:58:15.538 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.540 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=100, clusterName=obcluster, tenantName=sys, dbUser=root 2024-03-13 15:58:15.540 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.o.p.o.i.OasSessionStatServiceImpl : begin to get deadlock of obcluster1 2024-03-13 15:58:15.540 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.540 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.541 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.541 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.541 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.541 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2024-03-13 15:58:15.542 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY, args: [] 2024-03-13 15:58:15.543 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:15.543 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.o.p.o.i.OasSessionStatServiceImpl : finish fetching tenant session data of obcluster 2024-03-13 15:58:15.543 INFO 1433467 --- [ForkJoinPool-11-worker-9,2e5169d1b35d4c79,4788bc519b61] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching tenant session stat of obcluster 2024-03-13 15:58:15.543 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.575 INFO 1433467 --- [ForkJoinPool-11-worker-7,2e5169d1b35d4c79,5f72d35a0221] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2024-03-13 15:58:15.581 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=100, clusterName=obcluster1, tenantName=tenant1, dbUser=root 2024-03-13 15:58:15.582 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.582 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.582 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.582 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select outline_name as outlineName,outline_id, outline_content, database_id, database_name, visible_signature, sql_text, concurrent_num, limit_target, create_time as gmt_create, modify_time from DBA_OB_CONCURRENT_LIMIT_SQL 2024-03-13 15:58:15.583 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=100, clusterName=obcluster1, tenantName=sys, dbUser=root 2024-03-13 15:58:15.583 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.583 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.583 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.583 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.583 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.584 INFO 1433467 --- [ForkJoinPool-11-worker-0,2e5169d1b35d4c79,701135b46c56] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select database_id, tenant_id, outline_id, outline_name as outlineName, outline_content, sql_text, sql_id, outline_target, create_time as gmt_create, modify_time from DBA_OB_OUTLINES 2024-03-13 15:58:15.584 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY, args: [] 2024-03-13 15:58:15.589 INFO 1433467 --- [pool-10-thread-11,2e5169d1b35d4c79,1898e649d4a6] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching outline history 2024-03-13 15:58:15.593 INFO 1433467 --- [ForkJoinPool.commonPool-worker-3,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.594 INFO 1433467 --- [ForkJoinPool.commonPool-worker-3,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.594 INFO 1433467 --- [ForkJoinPool.commonPool-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.594 INFO 1433467 --- [ForkJoinPool.commonPool-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(60000000) */ time_to_usec(t1.MODIFY_TIME) AS UPDATE_TIME, t1.RESOURCE_POOL_ID, t1.NAME, t1.UNIT_COUNT, t1.UNIT_CONFIG_ID, t1.ZONE_LIST, t1.TENANT_ID, t1.REPLICA_TYPE, t2.NAME AS UNIT_CONFIG_NAME, t2.MAX_CPU, t2.MIN_CPU, t2.MEMORY_SIZE AS MAX_MEMORY, t2.MEMORY_SIZE AS MIN_MEMORY, t2.LOG_DISK_SIZE, t2.MAX_IOPS, t2.MIN_IOPS, t2.IOPS_WEIGHT FROM oceanbase.DBA_OB_RESOURCE_POOLS AS t1 JOIN oceanbase.DBA_OB_UNIT_CONFIGS AS t2 ON t1.UNIT_CONFIG_ID = t2.UNIT_CONFIG_ID AND t1.tenant_id = ?, args: [1002] 2024-03-13 15:58:15.595 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=100, clusterName=obcluster1, tenantName=tenant1, dbUser=root 2024-03-13 15:58:15.595 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.596 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.596 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.596 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT event_id, svr_ip, svr_port, report_time, cycle_idx as cycle_index, cycle_size, role, visitor, object as queryResource, extra_name1, extra_value1 from DBA_OB_DEADLOCK_EVENT_HISTORY, args: [] 2024-03-13 15:58:15.598 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.o.p.o.i.OasSessionStatServiceImpl : finish fetching tenant session data of obcluster1 2024-03-13 15:58:15.598 INFO 1433467 --- [ForkJoinPool-11-worker-2,2e5169d1b35d4c79,23ea34616391] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching tenant session stat of obcluster1 2024-03-13 15:58:15.603 INFO 1433467 --- [pool-10-thread-9,2e5169d1b35d4c79,6669334f847d] c.o.o.p.oas.internal.OasStatServiceImpl : finish fetching session stat 2024-03-13 15:58:15.631 INFO 1433467 --- [ForkJoinPool.commonPool-worker-3,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:15.632 INFO 1433467 --- [ForkJoinPool.commonPool-worker-3,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.632 INFO 1433467 --- [ForkJoinPool.commonPool-worker-3,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:15.632 INFO 1433467 --- [ForkJoinPool.commonPool-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:15.633 INFO 1433467 --- [ForkJoinPool.commonPool-worker-3,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(5000000) */ t1.UNIT_ID, t1.RESOURCE_POOL_ID, t1.UNIT_CONFIG_ID, t1.ZONE, t1.SVR_IP, t1.SVR_PORT, t1.STATUS, t2.REPLICA_TYPE, t2.TENANT_ID, t3.TENANT_NAME, t2.NAME AS RESOURCE_POOL_NAME, t2.MODIFY_TIME AS RESOURCE_POOL_UPDATE_TIME, t1.MIGRATE_FROM_SVR_IP, t1.MIGRATE_FROM_SVR_PORT, t1.MANUAL_MIGRATE FROM oceanbase.DBA_OB_UNITS AS t1 JOIN oceanbase.DBA_OB_RESOURCE_POOLS AS t2 ON t1.RESOURCE_POOL_ID = t2.RESOURCE_POOL_ID JOIN oceanbase.DBA_OB_TENANTS t3 ON t2.TENANT_ID = t3.TENANT_ID WHERE 1=1 AND t2.tenant_id = ?, args: [1002] 2024-03-13 15:58:15.932 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806978,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:16.112 INFO 1433467 --- [SimpleAsyncTaskExecutor-89806981,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:58:16.229 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:16.233 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:16.306 INFO 1433467 --- [pool-overview-container-9,f65c5c96a7034113,48f9e0bf31e6] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:16.306 INFO 1433467 --- [pool-overview-container-9,f65c5c96a7034113,48f9e0bf31e6] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:16.307 INFO 1433467 --- [pool-overview-container-9,f65c5c96a7034113,48f9e0bf31e6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:16.307 INFO 1433467 --- [pool-overview-container-9,f65c5c96a7034113,48f9e0bf31e6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1 ORDER BY start_time DESC) 2024-03-13 15:58:16.779 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:58:16.779255+08:00, lastProcessTime=2024-03-13T15:58:06.746295+08:00 2024-03-13 15:58:16.922 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:58:16.779255+08:00 2024-03-13 15:58:17.048 INFO 1433467 --- [pool-ocp-schedules-5,a9bc8affc7d84dda,699755c549df] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:58:17.933 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807002,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:18.378 INFO 1433467 --- [pool-overview-container-9,f65c5c96a7034113,48f9e0bf31e6] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:18.379 INFO 1433467 --- [pool-overview-container-9,f65c5c96a7034113,48f9e0bf31e6] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:18.379 INFO 1433467 --- [pool-overview-container-9,f65c5c96a7034113,48f9e0bf31e6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:18.379 INFO 1433467 --- [pool-overview-container-9,f65c5c96a7034113,48f9e0bf31e6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT * FROM (SELECT * FROM (SELECT job_id, incarnation, job.tenant_id as tenant_id, job.backup_set_id as backup_set_id, backup_type, path AS backup_dest, start_timestamp as start_time, end_timestamp as end_time, now(6) as check_time, status, comment, description, 'TENANT' as backup_level, file.MIN_RESTORE_SCN_DISPLAY as snapshot_version_time FROM ( SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, NULL AS end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOBS UNION SELECT job_id, incarnation, tenant_id, backup_set_id, backup_type, path, start_timestamp, end_timestamp, status, comment, description FROM CDB_OB_BACKUP_JOB_HISTORY ) AS job LEFT JOIN (SELECT tenant_id AS tid, backup_set_id AS sid, min_restore_scn_display FROM CDB_OB_BACKUP_SET_FILES) AS file ON job.tenant_id = file.tid AND job.backup_set_id = file.sid) WHERE tenant_id != 1 ORDER BY start_time DESC) 2024-03-13 15:58:19.233 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:19.234 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:19.275 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807017,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:58:19.286 INFO 1433467 --- [alarm-detect-9,c3d3825b973f4e1a,4c270d21bb9c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000913, data is empty. 2024-03-13 15:58:19.286 INFO 1433467 --- [alarm-detect-11,ee00ec4bb5204ec9,86c6dd97fe29] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000913, data is empty. 2024-03-13 15:58:19.288 INFO 1433467 --- [alarm-detect-26,4fe34925a0d34078,6f590e02d91b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000913, data is empty. 2024-03-13 15:58:19.290 INFO 1433467 --- [alarm-detect-11,ee00ec4bb5204ec9,86c6dd97fe29] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000914, data is empty. 2024-03-13 15:58:19.291 INFO 1433467 --- [alarm-detect-9,c3d3825b973f4e1a,4c270d21bb9c] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000914, data is empty. 2024-03-13 15:58:19.292 INFO 1433467 --- [alarm-detect-26,4fe34925a0d34078,6f590e02d91b] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000914, data is empty. 2024-03-13 15:58:19.314 WARN 1433467 --- [pool-scanner-1609809170-1,a9ad50690cab4ce9,9cbae64113a0] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:58:19.450 INFO 1433467 --- [alarm-detect-7,b5236ce0c27f4d2d,0d0e1f89a567] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316665 ~ 1710316689 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316680=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:58:19.451 INFO 1433467 --- [alarm-detect-7,b5236ce0c27f4d2d,0d0e1f89a567] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316665 ~ 1710316689 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316680=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:58:19.451 INFO 1433467 --- [alarm-detect-7,b5236ce0c27f4d2d,0d0e1f89a567] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316665 ~ 1710316689 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316680=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:58:19.451 INFO 1433467 --- [alarm-detect-7,b5236ce0c27f4d2d,0d0e1f89a567] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316665 ~ 1710316689 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316680=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:58:19.451 INFO 1433467 --- [alarm-detect-7,b5236ce0c27f4d2d,0d0e1f89a567] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316665 ~ 1710316689 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316680=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:58:19.451 INFO 1433467 --- [alarm-detect-7,b5236ce0c27f4d2d,0d0e1f89a567] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316665 ~ 1710316689 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316680=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:58:19.933 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807025,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:20.577 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,60018f9944fa45d2,2f2716b82914] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.144, traceId=60018f9944fa45d2, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:58:20.597 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-191,60018f9944fa45d2,2f2716b82914] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.144, traceId=60018f9944fa45d2, duration=21 ms] 2024-03-13 15:58:20.824 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:57:20.0, nextRun=2024-03-13 15:58:20.0, currentDate=Wed Mar 13 15:58:20 CST 2024 2024-03-13 15:58:20.824 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999996, name=Collect all cluster compaction info, templateName=CollectClusterCompactionSchedule, scheduleRule=20 * * * * *, lastRunTime=2024-03-13 15:57:20.0, nextRunTime=2024-03-13 15:58:20.0} 2024-03-13 15:58:20.829 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Collect all cluster compaction info, nodeCount=1, taskCount=1 2024-03-13 15:58:20.829 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:20.830 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Collect all cluster compaction info, taskDefinitionId=999996, state=RUNNING, operation=EXECUTE} 2024-03-13 15:58:21.263 INFO 1433467 --- [pool-subtask-coordinator1,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016777, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:21.287 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,324f31d93c5e] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016777, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014423, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:21.259+08:00, sub_task_instance_name=Collect compaction for all clusters, sub_task_instance_id=10016777}, listMap={}}, executor=10.10.10.148 2024-03-13 15:58:21.289 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,324f31d93c5e] c.o.o.s.t.b.c.AbstractAllClusterTask : Collect compaction for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:58:21.289 INFO 1433467 --- [ForkJoinPool-1386603-worker-1,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:58:21.289 INFO 1433467 --- [ForkJoinPool-1386603-worker-2,,] s.t.b.c.CollectAllClustersCompactionTask : Ob version >= 4.0.0.0, skip collect compaction 2024-03-13 15:58:21.297 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,324f31d93c5e] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016777, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:21.484 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014423, status:SUCCESS 2024-03-13 15:58:21.488 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014423, status:SUCCESS 2024-03-13 15:58:21.934 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807048,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:22.099 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,98d7de5ab8d94366,27077be0262b] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.148, traceId=98d7de5ab8d94366, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:58:22.106 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-110,98d7de5ab8d94366,27077be0262b] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.148, traceId=98d7de5ab8d94366, duration=8 ms] 2024-03-13 15:58:22.234 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:22.235 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:23.934 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807071,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:25.236 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:25.237 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:25.934 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807094,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:26.112 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807097,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:58:26.728 INFO 1433467 --- [pool-Check obproxy13,540dc03680714924,1ac125b66ad5] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:58:26.728 INFO 1433467 --- [pool-Check obproxy11,540dc03680714924,3863860f5385] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:58:26.737 INFO 1433467 --- [pool-Check obproxy2,540dc03680714924,ad82ece4401d] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:58:26.738 INFO 1433467 --- [pool-Check obproxy3,540dc03680714924,f4603240ba8d] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:58:26.781 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:58:26.780479+08:00, lastProcessTime=2024-03-13T15:58:16.779255+08:00 2024-03-13 15:58:26.901 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:58:26.780479+08:00 2024-03-13 15:58:27.056 INFO 1433467 --- [pool-ocp-schedules-11,a4862fe5dec2474a,94fd7afbee75] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:58:27.935 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807118,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:28.237 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:28.238 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:29.274 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807133,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:58:29.330 WARN 1433467 --- [pool-scanner-1609809776-1,b93e3567eee6437c,9a593240cd2a] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:58:29.413 INFO 1433467 --- [alarm-detect-20,29e00e0a139a4ce5,daa0b044bcff] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000914, data is empty. 2024-03-13 15:58:29.415 INFO 1433467 --- [alarm-detect-20,29e00e0a139a4ce5,daa0b044bcff] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000914, data is empty. 2024-03-13 15:58:29.469 INFO 1433467 --- [alarm-detect-20,9f74459f97f346ea,309d0945791e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316680 ~ 1710316699 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316695=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.148, host_name=ShuHeCrmSqlFour, ob_cluster_id=1, ob_cluster_name=obcluster, obzone=zone1, svr_ip=10.10.10.148})]}, matched=true, message=null, level=2) 2024-03-13 15:58:29.469 INFO 1433467 --- [alarm-detect-20,9f74459f97f346ea,309d0945791e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316680 ~ 1710316699 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316695=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:58:29.469 INFO 1433467 --- [alarm-detect-20,9f74459f97f346ea,309d0945791e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316680 ~ 1710316699 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316695=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:58:29.469 INFO 1433467 --- [alarm-detect-20,9f74459f97f346ea,309d0945791e] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316680 ~ 1710316699 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316695=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:58:29.935 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807142,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:30.148 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:57:31.0, nextRun=2024-03-13 15:58:30.0, currentDate=Wed Mar 13 15:58:30 CST 2024 2024-03-13 15:58:30.148 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999990, name=Sync tenant information, templateName=TenantSyncSchedule, scheduleRule=30 0/1 * * * ?, lastRunTime=2024-03-13 15:57:31.0, nextRunTime=2024-03-13 15:58:30.0} 2024-03-13 15:58:30.153 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Sync tenant information, nodeCount=1, taskCount=2 2024-03-13 15:58:30.153 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:30.154 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:30.154 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Sync tenant information, taskDefinitionId=999990, state=RUNNING, operation=EXECUTE} 2024-03-13 15:58:30.166 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.engine.util.SpringScheduler : Schedule time arrived, lastRun=2024-03-13 15:58:00.0, nextRun=2024-03-13 15:58:30.0, currentDate=Wed Mar 13 15:58:30 CST 2024 2024-03-13 15:58:30.166 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Schedule ready task, entity=TaskDefinitionEntity{id=999997, name=Check obproxy, templateName=CheckObproxyServerSchedule, scheduleRule=*/30 * * * * *, lastRunTime=2024-03-13 15:58:00.0, nextRunTime=2024-03-13 15:58:30.0} 2024-03-13 15:58:30.168 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.model.TaskInstanceEntity : generate tasks, templateName=Check obproxy, nodeCount=1, taskCount=1 2024-03-13 15:58:30.168 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.m.r.t.m.SubtaskInstanceEntity : Set state for subtask: null, current state: PENDING, new state: READY 2024-03-13 15:58:30.168 INFO 1433467 --- [pool-task-coordinator1,f47a761474a84dce,975a8653260a] c.o.o.c.t.e.c.worker.ReadyTaskWorker : Save task instance, entity=TaskInstanceEntity{id=null, name=Check obproxy, taskDefinitionId=999997, state=RUNNING, operation=EXECUTE} 2024-03-13 15:58:30.362 INFO 1433467 --- [pool-subtask-coordinator1,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016778, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:30.379 INFO 1433467 --- [pool-subtask-coordinator1,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016779, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:30.387 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016778, context=Context{parallelIdx=0, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014424, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:30.359+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016778}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:58:30.392 INFO 1433467 --- [pool-subtask-coordinator1,,] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016780, operation:EXECUTE, state: RUNNING 2024-03-13 15:58:30.402 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016779, context=Context{parallelIdx=1, stringMap={subtask_splitter=cluster_ids, task_instance_id=10014424, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:30.377+08:00, sub_task_instance_name=Sync all tenant information, sub_task_instance_id=10016779}, listMap={cluster_ids=[1, 2]}}, executor=10.10.10.148 2024-03-13 15:58:30.411 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,7c4a0dc7684b] c.o.o.c.t.e.runner.JavaSubtaskRunner : Run subtask, id=10016780, context=Context{parallelIdx=-1, stringMap={task_instance_id=10014425, task_operation=execute, latest_execution_start_time=2024-03-13T15:58:30.390+08:00, sub_task_instance_name=Check obproxy connection status, sub_task_instance_id=10016780}, listMap={}}, executor=10.10.10.148 2024-03-13 15:58:30.422 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,8d6013a94d0f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:58:30.423 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,99a581120c6c] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:58:30.423 INFO 1433467 --- [pool-Check obproxy10,f47a761474a84dce,d3c3f2935fef] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:58:30.424 INFO 1433467 --- [pool-Check obproxy8,f47a761474a84dce,bf39098bd399] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:58:30.437 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.438 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:58:30.438 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.438 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.439 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:58:30.444 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.444 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.444 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:58:30.448 INFO 1433467 --- [pool-Check obproxy4,f47a761474a84dce,8d6013a94d0f] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 3 success, update last active time to now 2024-03-13 15:58:30.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.HintQueryer : add hints for sql, hints=[query_timeout(60000000)], after=SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:58:30.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.449 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ query_timeout(60000000) */ TENANT_ID, TENANT_NAME, TENANT_TYPE, PRIMARY_ZONE, LOCALITY, COMPATIBILITY_MODE, STATUS, IN_RECYCLEBIN, (CASE WHEN LOCKED = 'YES' THEN 1 ELSE 0 END) AS LOCKED, TIMESTAMPDIFF(SECOND, CREATE_TIME, now()) AS exist_seconds , ARBITRATION_SERVICE_STATUS, REPLACE(`SWITCHOVER_STATUS`, ' ', '_') as SWITCHOVER_STATUS, LOG_MODE, SYNC_SCN, RECOVERY_UNTIL_SCN, TENANT_ROLE FROM oceanbase.DBA_OB_TENANTS WHERE TENANT_TYPE IN ('SYS', 'USER') 2024-03-13 15:58:30.451 INFO 1433467 --- [pool-Check obproxy8,f47a761474a84dce,bf39098bd399] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 4 success, update last active time to now 2024-03-13 15:58:30.454 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.454 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.454 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:58:30.455 INFO 1433467 --- [pool-Check obproxy10,f47a761474a84dce,d3c3f2935fef] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 6 success, update last active time to now 2024-03-13 15:58:30.456 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.456 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.456 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:58:30.460 INFO 1433467 --- [pool-Check obproxy14,f47a761474a84dce,99a581120c6c] c.o.o.o.internal.ObproxyServiceImpl : check obproxy 5 success, update last active time to now 2024-03-13 15:58:30.462 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.462 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.462 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:58:30.464 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.464 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.464 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:58:30.472 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.472 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.472 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.472 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.472 INFO 1433467 --- [pool-subtask-executor25,f47a761474a84dce,7c4a0dc7684b] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016780, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:30.472 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:58:30.472 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:58:30.478 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.478 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.478 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:58:30.492 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.492 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.492 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.492 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.492 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:58:30.498 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.498 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.499 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.499 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.499 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:58:30.501 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.501 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.501 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.501 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.502 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_TENANT_LOCALITY] 2024-03-13 15:58:30.504 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.504 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.505 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.505 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.505 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:58:30.511 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.512 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.512 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.512 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.512 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:58:30.519 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.520 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.520 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.520 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.521 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:58:30.524 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.524 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.524 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.524 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.525 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:58:30.530 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.530 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.531 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.531 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.531 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:58:30.538 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.539 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.539 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.539 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.540 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:58:30.540 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=2 2024-03-13 15:58:30.543 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.543 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.544 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.544 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.544 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_TENANT_LOCALITY] 2024-03-13 15:58:30.544 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:58:30.554 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.555 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.556 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.556 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.556 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.556 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.557 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, SHRINK_RESOURCE_POOL_UNIT_NUM] 2024-03-13 15:58:30.558 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.558 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.559 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:58:30.566 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.566 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.567 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.567 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.567 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, JOB_ID, JOB_TYPE, JOB_STATUS, PROGRESS FROM oceanbase.DBA_OB_TENANT_JOBS WHERE TENANT_ID = ? AND JOB_TYPE = ? ORDER BY JOB_ID DESC LIMIT 1, args: [1002, ALTER_RESOURCE_TENANT_UNIT_NUM] 2024-03-13 15:58:30.569 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648001] sync all tenant whitelist start, clusterId=1 2024-03-13 15:58:30.572 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648111] default whitelist refreshed, defaultWhitelist=10.10.10.148 2024-03-13 15:58:30.575 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=4, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:58:30.578 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=4, userWhitelist=% 2024-03-13 15:58:30.578 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:30.578 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.579 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:58:30.586 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=3, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:58:30.588 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=3, userWhitelist=% 2024-03-13 15:58:30.589 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=2 2024-03-13 15:58:30.589 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:58:30.594 INFO 1433467 --- [pool-subtask-executor16,f47a761474a84dce,67a39bad9179] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016779, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:30.594 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:30.595 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.597 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.597 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.597 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1] 2024-03-13 15:58:30.605 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=1, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:58:30.606 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=1, userWhitelist=% 2024-03-13 15:58:30.607 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:30.607 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:30.607 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT TENANT_ID, NAME, VALUE FROM oceanbase.CDB_OB_SYS_VARIABLES WHERE TENANT_ID = ?, args: [1002] 2024-03-13 15:58:30.613 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648201] user whitelist calculated, tenantId=2, realWhitelist=%, defaultWhitelist=127.0.0.1,10.10.10.148, userWhitelist=% 2024-03-13 15:58:30.614 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648202] user whitelist updated, tenantId=2, userWhitelist=% 2024-03-13 15:58:30.614 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.o.i.t.TenantWhitelistServiceImpl : [OCP648002] sync all tenant whitelist done, clusterId=1 2024-03-13 15:58:30.614 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.s.t.b.t.SyncAllTenantInfoTask : [OCP606] sync tenants success, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:58:30.617 INFO 1433467 --- [pool-subtask-executor28,f47a761474a84dce,03103110fe55] c.o.o.c.t.e.c.w.subtask.SubtaskAccessor : Set state for subtask: 10016778, operation:EXECUTE, state: SUCCESSFUL 2024-03-13 15:58:30.644 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014424, status:SUCCESS 2024-03-13 15:58:30.645 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014424, status:SUCCESS 2024-03-13 15:58:30.663 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : begin update audit event, taskId:10014425, status:SUCCESS 2024-03-13 15:58:30.664 INFO 1433467 --- [pool-task-coordinator2,e91b454d298c41ec,2f75a2440768] c.o.ocp.service.audit.AuditEventService : audit event not found, taskId:10014425, status:SUCCESS 2024-03-13 15:58:31.239 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:31.240 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:31.936 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807165,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:33.269 INFO 1433467 --- [pool-ocp-schedules-16,2dbac2af8f754028,e0fbea87a512] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:58:33.281 INFO 1433467 --- [pool-ocp-schedules-16,2dbac2af8f754028,e0fbea87a512] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:58:33.282 INFO 1433467 --- [pool-ocp-schedules-16,2dbac2af8f754028,e0fbea87a512] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:58:33.936 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807188,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:34.240 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:34.241 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:35.937 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807210,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:36.113 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807213,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:58:36.746 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:58:36.746029+08:00, lastProcessTime=2024-03-13T15:58:26.780479+08:00 2024-03-13 15:58:36.855 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:58:36.746029+08:00 2024-03-13 15:58:37.060 INFO 1433467 --- [pool-ocp-schedules-9,f5c052b015144c61,71f7e30d8740] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:58:37.241 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:37.242 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:37.937 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807234,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:38.375 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,62c3fc66e04f419b,b60ba0add23f] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.143, traceId=62c3fc66e04f419b, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:58:38.382 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-140,62c3fc66e04f419b,b60ba0add23f] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.143, traceId=62c3fc66e04f419b, duration=8 ms] 2024-03-13 15:58:38.852 INFO 1433467 --- [ForkJoinPool.commonPool-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:38.856 INFO 1433467 --- [ForkJoinPool.commonPool-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:38.856 INFO 1433467 --- [ForkJoinPool.commonPool-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:38.856 INFO 1433467 --- [ForkJoinPool.commonPool-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT SVR_IP, SVR_PORT, ZONE, CPU_CAPACITY, CPU_CAPACITY_MAX, CPU_ASSIGNED, CPU_ASSIGNED_MAX, MEM_CAPACITY, MEM_CAPACITY AS MEM_CAPACITY_MAX, MEM_ASSIGNED, MEM_ASSIGNED AS MEM_ASSIGNED_MAX, LOG_DISK_CAPACITY, LOG_DISK_ASSIGNED, LOG_DISK_IN_USE, DATA_DISK_CAPACITY, DATA_DISK_IN_USE FROM oceanbase.GV$OB_SERVERS 2024-03-13 15:58:38.860 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:38.861 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:38.861 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:38.861 INFO 1433467 --- [ForkJoinPool.commonPool-worker-4,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT SVR_IP, SVR_PORT, ZONE, CPU_CAPACITY, CPU_CAPACITY_MAX, CPU_ASSIGNED, CPU_ASSIGNED_MAX, MEM_CAPACITY, MEM_CAPACITY AS MEM_CAPACITY_MAX, MEM_ASSIGNED, MEM_ASSIGNED AS MEM_ASSIGNED_MAX, LOG_DISK_CAPACITY, LOG_DISK_ASSIGNED, LOG_DISK_IN_USE, DATA_DISK_CAPACITY, DATA_DISK_IN_USE FROM oceanbase.GV$OB_SERVERS 2024-03-13 15:58:39.255 INFO 1433467 --- [pool-ocp-schedules-7,b0388334916d4611,330a491e5e8d] c.o.o.s.task.scheduled.ClusterSchedules : Check cluster status schedule. 2024-03-13 15:58:39.256 INFO 1433467 --- [pool-ocp-schedules-18,a498c457b52a4240,a666d5b5ede2] c.o.o.s.task.scheduled.ClusterSchedules : Sync cluster info schedule. 2024-03-13 15:58:39.256 INFO 1433467 --- [pool-ocp-schedules-9,1bd120db3e2c41ce,f750fb87f715] c.o.o.s.task.scheduled.ClusterSchedules : Check standby cluster info schedule. 2024-03-13 15:58:39.257 INFO 1433467 --- [pool-ocp-schedules-7,b0388334916d4611,330a491e5e8d] c.o.o.s.t.b.c.AbstractAllClusterTask : Check status for all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:58:39.258 INFO 1433467 --- [pool-ocp-schedules-18,a498c457b52a4240,a666d5b5ede2] c.o.o.s.t.b.c.AbstractAllClusterTask : Sync all clusters executed completed, tryCount=2, successCount=0 2024-03-13 15:58:39.259 INFO 1433467 --- [pool-ocp-schedules-9,1bd120db3e2c41ce,f750fb87f715] c.o.o.s.t.b.c.AbstractAllClusterTask : Check all standby clusters sync delay time executed completed, tryCount=2, successCount=0 2024-03-13 15:58:39.273 INFO 1433467 --- [pool-ocp-schedules-11,e0b9d4acdf4e4ef1,b3444c1cd6a7] c.o.o.s.task.scheduled.ClusterSchedules : Check all arbitration services 2024-03-13 15:58:39.275 INFO 1433467 --- [pool-ocp-schedules-19,0c8a0212a2a445b4,9f60b04028c8] c.o.o.s.task.scheduled.ClusterSchedules : Check all standby tenant 2024-03-13 15:58:39.275 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807250,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:58:39.281 INFO 1433467 --- [alarm-detect-5,6c8fbc688cd8432c,28087582a469] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000914, data is empty. 2024-03-13 15:58:39.282 INFO 1433467 --- [alarm-detect-20,442563a7e29245ce,be2850028107] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000914, data is empty. 2024-03-13 15:58:39.285 INFO 1433467 --- [alarm-detect-5,6c8fbc688cd8432c,28087582a469] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_long_trans_exist,sql_alarm_type-trans_stat,trans_state-long, startId: 9000914, data is empty. 2024-03-13 15:58:39.285 INFO 1433467 --- [alarm-detect-4,ad3a26bcaab44bd2,6e791eeed88d] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000914, data is empty. 2024-03-13 15:58:39.286 INFO 1433467 --- [alarm-detect-20,442563a7e29245ce,be2850028107] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_expired_trans_exist,is_xa_trans-0,sql_alarm_type-trans_stat,trans_state-pending, startId: 9000914, data is empty. 2024-03-13 15:58:39.288 INFO 1433467 --- [alarm-detect-4,ad3a26bcaab44bd2,6e791eeed88d] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_large_trans_exist,sql_alarm_type-trans_stat, startId: 9000914, data is empty. 2024-03-13 15:58:39.293 INFO 1433467 --- [ForkJoinPool-1386604-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.293 INFO 1433467 --- [ForkJoinPool-1386604-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.293 INFO 1433467 --- [ForkJoinPool-1386604-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.293 INFO 1433467 --- [ForkJoinPool-1386604-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:58:39.294 INFO 1433467 --- [ForkJoinPool-1386604-worker-1,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=2, obVersion=4.2.1.4 2024-03-13 15:58:39.294 INFO 1433467 --- [pool-ocp-schedules-17,9a2ca15aada1438f,c44cae22e906] c.o.o.s.task.scheduled.ClusterSchedules : Check host status schedule. 2024-03-13 15:58:39.294 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:58:39.294 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:58:39.294 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:58:39.294 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:58:39.294 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:58:39.295 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:58:39.295 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:58:39.297 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:58:39.297 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:58:39.298 INFO 1433467 --- [ForkJoinPool-1386604-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.298 INFO 1433467 --- [ForkJoinPool-1386604-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.298 INFO 1433467 --- [ForkJoinPool-1386604-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.298 INFO 1433467 --- [ForkJoinPool-1386604-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:58:39.299 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:58:39.299 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:58:39.299 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:58:39.300 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:58:39.300 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:58:39.301 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:58:39.302 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:58:39.302 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:58:39.302 INFO 1433467 --- [ForkJoinPool-1386604-worker-2,,] c.o.o.s.t.b.c.CheckClusterStatusTask : check cluster status okay, clusterId=1, obVersion=4.2.1.4 2024-03-13 15:58:39.303 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:58:39.304 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:58:39.305 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:58:39.307 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:58:39.309 INFO 1433467 --- [pool-ocp-schedules-12,70e2d1b033b4467c,4c27fadb1018] c.o.o.s.task.scheduled.ClusterSchedules : Check host agent status schedule. 2024-03-13 15:58:39.311 INFO 1433467 --- [pool-ocp-schedules-10,959ecdb8a2d34e22,0ca23f43cc35] c.o.o.s.task.scheduled.ClusterSchedules : Check host clockdiff schedule. 2024-03-13 15:58:39.319 INFO 1433467 --- [pool-Check host task for clockdiff12,959ecdb8a2d34e22,6be9cbc70c10] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.142], timeoutMillis=20000 2024-03-13 15:58:39.319 INFO 1433467 --- [pool-Check host task for clockdiff16,959ecdb8a2d34e22,b6862860ccfe] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.143], timeoutMillis=20000 2024-03-13 15:58:39.319 INFO 1433467 --- [pool-Check host task for clockdiff15,959ecdb8a2d34e22,070791fb7dbb] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.144], timeoutMillis=20000 2024-03-13 15:58:39.319 INFO 1433467 --- [pool-Check host task for clockdiff17,959ecdb8a2d34e22,eda9df0f7ef7] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.148], timeoutMillis=20000 2024-03-13 15:58:39.319 INFO 1433467 --- [pool-Check host task for clockdiff20,959ecdb8a2d34e22,ddc4f7b25baf] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.146], timeoutMillis=20000 2024-03-13 15:58:39.321 INFO 1433467 --- [pool-Check host task for clockdiff18,959ecdb8a2d34e22,ada2f04c51e4] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.147], timeoutMillis=20000 2024-03-13 15:58:39.321 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.321 INFO 1433467 --- [pool-Check host task for clockdiff30,959ecdb8a2d34e22,b42c140b275f] c.o.ocp.core.util.LocalCommandExecutor : Execute local command, executable=clockdiff, args=[10.10.10.145], timeoutMillis=20000 2024-03-13 15:58:39.322 INFO 1433467 --- [pool-Check host task for agent12,70e2d1b033b4467c,ba936f0e9410] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=7, ip=10.10.10.142, originalStatus=ONLINE 2024-03-13 15:58:39.322 INFO 1433467 --- [pool-Check host task for agent26,70e2d1b033b4467c,4fac9f0f8b56] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=5, ip=10.10.10.143, originalStatus=ONLINE 2024-03-13 15:58:39.322 INFO 1433467 --- [pool-Check host task for agent10,70e2d1b033b4467c,2f4079455344] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=4, ip=10.10.10.148, originalStatus=ONLINE 2024-03-13 15:58:39.322 INFO 1433467 --- [pool-Check host task for agent13,70e2d1b033b4467c,b96e3723c53a] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=3, ip=10.10.10.146, originalStatus=ONLINE 2024-03-13 15:58:39.322 INFO 1433467 --- [pool-Check host task for agent27,70e2d1b033b4467c,87897053c94d] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=2, ip=10.10.10.147, originalStatus=ONLINE 2024-03-13 15:58:39.322 INFO 1433467 --- [pool-Check host task for agent19,70e2d1b033b4467c,9c2f92b75b33] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=1, ip=10.10.10.145, originalStatus=ONLINE 2024-03-13 15:58:39.322 INFO 1433467 --- [pool-Check host task for agent14,70e2d1b033b4467c,da6b11bd3385] c.o.o.c.h.manager.HostCheckManagerImpl : Check host agent status, hostId=6, ip=10.10.10.144, originalStatus=ONLINE 2024-03-13 15:58:39.322 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.323 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.326 INFO 1433467 --- [pool-Check host task for agent13,70e2d1b033b4467c,b96e3723c53a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:58:39.327 INFO 1433467 --- [pool-Check host task for agent10,70e2d1b033b4467c,2f4079455344] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:58:39.327 INFO 1433467 --- [pool-Check host task for agent14,70e2d1b033b4467c,da6b11bd3385] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:58:39.327 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:58:39.327 INFO 1433467 --- [pool-Check host task for agent26,70e2d1b033b4467c,4fac9f0f8b56] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:58:39.328 INFO 1433467 --- [pool-Check host task for agent13,70e2d1b033b4467c,b96e3723c53a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:58:39.329 INFO 1433467 --- [pool-Check host task for agent19,70e2d1b033b4467c,9c2f92b75b33] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:58:39.329 INFO 1433467 --- [pool-Check host task for agent10,70e2d1b033b4467c,2f4079455344] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:58:39.329 INFO 1433467 --- [pool-Check host task for agent14,70e2d1b033b4467c,da6b11bd3385] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:58:39.330 INFO 1433467 --- [pool-Check host task for agent26,70e2d1b033b4467c,4fac9f0f8b56] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:58:39.330 INFO 1433467 --- [pool-Check host task for agent19,70e2d1b033b4467c,9c2f92b75b33] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:58:39.332 INFO 1433467 --- [pool-Check host task for agent27,70e2d1b033b4467c,87897053c94d] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:58:39.346 INFO 1433467 --- [pool-Check host task for agent27,70e2d1b033b4467c,87897053c94d] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:58:39.383 INFO 1433467 --- [pool-Check host task for agent14,70e2d1b033b4467c,da6b11bd3385] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/agent/status, params:null 2024-03-13 15:58:39.384 INFO 1433467 --- [pool-Check host task for agent26,70e2d1b033b4467c,4fac9f0f8b56] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/agent/status, params:null 2024-03-13 15:58:39.385 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/time, params:null 2024-03-13 15:58:39.386 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.143:62888/api/v1/time, params:null 2024-03-13 15:58:39.387 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/time, params:null 2024-03-13 15:58:39.387 INFO 1433467 --- [pool-Check host task for agent19,70e2d1b033b4467c,9c2f92b75b33] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.145:62888/api/v1/agent/status, params:null 2024-03-13 15:58:39.388 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=7, ip=10.10.10.142, timestamp=2024-03-13T15:58:39.387830152+08:00 2024-03-13 15:58:39.388 INFO 1433467 --- [pool-Check host task for agent27,70e2d1b033b4467c,87897053c94d] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/agent/status, params:null 2024-03-13 15:58:39.388 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=5, ip=10.10.10.143, timestamp=2024-03-13T15:58:39.387588550+08:00 2024-03-13 15:58:39.388 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=7, lastAvailableTime=2024-03-13T15:58:39.388+08:00 2024-03-13 15:58:39.388 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=5, lastAvailableTime=2024-03-13T15:58:39.388+08:00 2024-03-13 15:58:39.388 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=1, ip=10.10.10.145, timestamp=2024-03-13T15:58:39.388707011+08:00 2024-03-13 15:58:39.389 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.144:62888/api/v1/time, params:null 2024-03-13 15:58:39.390 INFO 1433467 --- [pool-Check host task for agent26,70e2d1b033b4467c,4fac9f0f8b56] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.143 2024-03-13 15:58:39.390 INFO 1433467 --- [pool-Check host task for agent26,70e2d1b033b4467c,4fac9f0f8b56] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.143 2024-03-13 15:58:39.390 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=1, lastAvailableTime=2024-03-13T15:58:39.389+08:00 2024-03-13 15:58:39.390 INFO 1433467 --- [pool-Check host task for agent26,70e2d1b033b4467c,4fac9f0f8b56] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.143 2024-03-13 15:58:39.390 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.s.compute.HostAgentDaoManager : Updated agent 5 last available time 2024-03-13T15:58:39.388+08:00 and status AVAILABLE 2024-03-13 15:58:39.390 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.s.compute.HostAgentDaoManager : Updated agent 9 last available time 2024-03-13T15:58:39.388+08:00 and status AVAILABLE 2024-03-13 15:58:39.390 INFO 1433467 --- [pool-Check host task for agent14,70e2d1b033b4467c,da6b11bd3385] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.144 2024-03-13 15:58:39.390 INFO 1433467 --- [pool-Check host task for agent14,70e2d1b033b4467c,da6b11bd3385] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.144 2024-03-13 15:58:39.390 INFO 1433467 --- [pool-Check host task for agent14,70e2d1b033b4467c,da6b11bd3385] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.144 2024-03-13 15:58:39.391 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=6, ip=10.10.10.144, timestamp=2024-03-13T15:58:39.390965711+08:00 2024-03-13 15:58:39.391 INFO 1433467 --- [pool-Check host task for agent19,70e2d1b033b4467c,9c2f92b75b33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.145 2024-03-13 15:58:39.391 INFO 1433467 --- [pool-Check host task for agent19,70e2d1b033b4467c,9c2f92b75b33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.145 2024-03-13 15:58:39.391 INFO 1433467 --- [pool-Check host task for agent19,70e2d1b033b4467c,9c2f92b75b33] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.145 2024-03-13 15:58:39.391 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.s.compute.HostAgentDaoManager : Updated agent 3 last available time 2024-03-13T15:58:39.389+08:00 and status AVAILABLE 2024-03-13 15:58:39.391 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=6, lastAvailableTime=2024-03-13T15:58:39.391+08:00 2024-03-13 15:58:39.391 INFO 1433467 --- [pool-Check host task for agent27,70e2d1b033b4467c,87897053c94d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.147 2024-03-13 15:58:39.391 INFO 1433467 --- [pool-Check host task for agent27,70e2d1b033b4467c,87897053c94d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.147 2024-03-13 15:58:39.391 INFO 1433467 --- [pool-Check host task for agent27,70e2d1b033b4467c,87897053c94d] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.147 2024-03-13 15:58:39.392 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.s.compute.HostAgentDaoManager : Updated agent 6 last available time 2024-03-13T15:58:39.391+08:00 and status AVAILABLE 2024-03-13 15:58:39.394 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=7 2024-03-13 15:58:39.394 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=6 2024-03-13 15:58:39.395 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=1 2024-03-13 15:58:39.395 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=5 2024-03-13 15:58:39.402 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/time, params:null 2024-03-13 15:58:39.404 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=3, ip=10.10.10.146, timestamp=2024-03-13T15:58:39.404143107+08:00 2024-03-13 15:58:39.405 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=3, lastAvailableTime=2024-03-13T15:58:39.404+08:00 2024-03-13 15:58:39.406 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.s.compute.HostAgentDaoManager : Updated agent 1 last available time 2024-03-13T15:58:39.404+08:00 and status AVAILABLE 2024-03-13 15:58:39.409 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=3 2024-03-13 15:58:39.417 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:58:39.418 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:58:39.422 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:58:39.422 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:58:39.423 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.147:62888/api/v1/time, params:null 2024-03-13 15:58:39.423 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:58:39.423 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:58:39.423 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=2, ip=10.10.10.147, timestamp=2024-03-13T15:58:39.423460371+08:00 2024-03-13 15:58:39.424 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=2, lastAvailableTime=2024-03-13T15:58:39.423+08:00 2024-03-13 15:58:39.426 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.s.compute.HostAgentDaoManager : Updated agent 2 last available time 2024-03-13T15:58:39.423+08:00 and status AVAILABLE 2024-03-13 15:58:39.429 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:58:39.429 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:58:39.430 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=2 2024-03-13 15:58:39.432 INFO 1433467 --- [pool-Check host task for agent13,70e2d1b033b4467c,b96e3723c53a] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.146:62888/api/v1/agent/status, params:null 2024-03-13 15:58:39.433 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:39.433 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/time, params:null 2024-03-13 15:58:39.435 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.h.manager.HostCheckManagerImpl : Check host status result. id=4, ip=10.10.10.148, timestamp=2024-03-13T15:58:39.435193406+08:00 2024-03-13 15:58:39.436 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.s.compute.HostAgentDaoManager : Updating OCP agent last available time with status as available, hostId=4, lastAvailableTime=2024-03-13T15:58:39.435+08:00 2024-03-13 15:58:39.437 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:39.437 INFO 1433467 --- [pool-Check host task for agent13,70e2d1b033b4467c,b96e3723c53a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.146 2024-03-13 15:58:39.437 INFO 1433467 --- [pool-Check host task for agent13,70e2d1b033b4467c,b96e3723c53a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.146 2024-03-13 15:58:39.437 INFO 1433467 --- [pool-Check host task for agent13,70e2d1b033b4467c,b96e3723c53a] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.146 2024-03-13 15:58:39.437 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.s.compute.HostAgentDaoManager : Updated agent 4 last available time 2024-03-13T15:58:39.435+08:00 and status AVAILABLE 2024-03-13 15:58:39.439 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:58:39.453 WARN 1433467 --- [pool-Check host task for clockdiff15,959ecdb8a2d34e22,070791fb7dbb] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.144], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.453 ERROR 1433467 --- [pool-Check host task for clockdiff15,959ecdb8a2d34e22,070791fb7dbb] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.144] 2024-03-13 15:58:39.453 WARN 1433467 --- [pool-Check host task for clockdiff15,959ecdb8a2d34e22,070791fb7dbb] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=6, ip=10.10.10.144, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.144 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.322 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.457 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.457 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.457 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:58:39.460 INFO 1433467 --- [pool-Check host task for agent12,70e2d1b033b4467c,ba936f0e9410] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:58:39.461 INFO 1433467 --- [pool-Check host task for agent12,70e2d1b033b4467c,ba936f0e9410] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:58:39.465 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000007 2024-03-13 15:58:39.472 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000022 2024-03-13 15:58:39.489 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:58:39.490 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:58:39.490 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.h.manager.HostCheckManagerImpl : Refreshing OCP agent process running info: hostId=4 2024-03-13 15:58:39.491 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:58:39.492 WARN 1433467 --- [pool-Check host task for clockdiff17,959ecdb8a2d34e22,eda9df0f7ef7] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.148], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.492 ERROR 1433467 --- [pool-Check host task for clockdiff17,959ecdb8a2d34e22,eda9df0f7ef7] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.148] 2024-03-13 15:58:39.492 WARN 1433467 --- [pool-Check host task for clockdiff17,959ecdb8a2d34e22,eda9df0f7ef7] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=4, ip=10.10.10.148, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.148 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.493 WARN 1433467 --- [pool-Check host task for clockdiff16,959ecdb8a2d34e22,b6862860ccfe] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.143], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.493 ERROR 1433467 --- [pool-Check host task for clockdiff16,959ecdb8a2d34e22,b6862860ccfe] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.143] 2024-03-13 15:58:39.493 WARN 1433467 --- [pool-Check host task for clockdiff16,959ecdb8a2d34e22,b6862860ccfe] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=5, ip=10.10.10.143, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.143 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.500 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:58:39.501 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:58:39.504 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:39.505 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:39.509 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:39.517 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:39.521 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000019 2024-03-13 15:58:39.524 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000025 2024-03-13 15:58:39.527 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000019 2024-03-13 15:58:39.533 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:39.533 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000025 2024-03-13 15:58:39.533 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.534 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.534 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.534 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:58:39.540 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000016 2024-03-13 15:58:39.547 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000016 2024-03-13 15:58:39.552 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:39.554 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000001 2024-03-13 15:58:39.556 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:39.557 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:39.558 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:39.559 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.560 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.560 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.560 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:58:39.561 WARN 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] idcName is empty, zoneName=zone1 2024-03-13 15:58:39.572 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000020 2024-03-13 15:58:39.572 INFO 1433467 --- [pool-Check host task for agent10,70e2d1b033b4467c,2f4079455344] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.148:62888/api/v1/agent/status, params:null 2024-03-13 15:58:39.575 INFO 1433467 --- [pool-Check host task for agent10,70e2d1b033b4467c,2f4079455344] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.148 2024-03-13 15:58:39.575 INFO 1433467 --- [pool-Check host task for agent10,70e2d1b033b4467c,2f4079455344] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.148 2024-03-13 15:58:39.575 INFO 1433467 --- [pool-Check host task for agent10,70e2d1b033b4467c,2f4079455344] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.148 2024-03-13 15:58:39.576 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000026 2024-03-13 15:58:39.576 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000001 2024-03-13 15:58:39.576 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.576 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.577 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:58:39.578 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000022 2024-03-13 15:58:39.578 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:58:39.578 WARN 1433467 --- [pool-Check host task for clockdiff20,959ecdb8a2d34e22,ddc4f7b25baf] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.146], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.579 ERROR 1433467 --- [pool-Check host task for clockdiff20,959ecdb8a2d34e22,ddc4f7b25baf] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.146] 2024-03-13 15:58:39.579 WARN 1433467 --- [pool-Check host task for clockdiff18,959ecdb8a2d34e22,ada2f04c51e4] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.147], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.579 ERROR 1433467 --- [pool-Check host task for clockdiff18,959ecdb8a2d34e22,ada2f04c51e4] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.147] 2024-03-13 15:58:39.579 WARN 1433467 --- [pool-Check host task for clockdiff20,959ecdb8a2d34e22,ddc4f7b25baf] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=3, ip=10.10.10.146, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.146 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.579 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000020 2024-03-13 15:58:39.579 WARN 1433467 --- [pool-Check host task for clockdiff18,959ecdb8a2d34e22,ada2f04c51e4] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=2, ip=10.10.10.147, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.147 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.579 WARN 1433467 --- [pool-Check host task for clockdiff12,959ecdb8a2d34e22,6be9cbc70c10] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.142], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.579 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_mgragent, skipPorts=false), params:null 2024-03-13 15:58:39.579 ERROR 1433467 --- [pool-Check host task for clockdiff12,959ecdb8a2d34e22,6be9cbc70c10] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.142] 2024-03-13 15:58:39.579 WARN 1433467 --- [pool-Check host task for clockdiff12,959ecdb8a2d34e22,6be9cbc70c10] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=7, ip=10.10.10.142, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.142 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.579 INFO 1433467 --- [pool-Check host task for agent12,70e2d1b033b4467c,ba936f0e9410] c.o.o.e.internal.template.HttpTemplate : GET request to agent, url:http://10.10.10.142:62888/api/v1/agent/status, params:null 2024-03-13 15:58:39.580 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000007 2024-03-13 15:58:39.585 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000026 2024-03-13 15:58:39.586 INFO 1433467 --- [pool-Check host task for agent12,70e2d1b033b4467c,ba936f0e9410] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_agentd status=running, hostIp=10.10.10.142 2024-03-13 15:58:39.586 INFO 1433467 --- [pool-Check host task for agent12,70e2d1b033b4467c,ba936f0e9410] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mgragent status=running, hostIp=10.10.10.142 2024-03-13 15:58:39.586 INFO 1433467 --- [pool-Check host task for agent12,70e2d1b033b4467c,ba936f0e9410] c.o.o.c.h.manager.HostCheckManagerImpl : Host agent ocp_mongent status=running, hostIp=10.10.10.142 2024-03-13 15:58:39.591 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000017 2024-03-13 15:58:39.593 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:39.594 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=4, name=zone1, idcId=4, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:39.594 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:49+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:58:39.597 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000017 2024-03-13 15:58:39.598 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:39.600 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:39.604 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:39.605 WARN 1433467 --- [pool-scanner-1609810456-1,7abc6366d0844765,c76dc65ae0e0] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:58:39.606 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.147:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:39.609 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=4, ip=10.10.10.148, port=2882, sqlPort=2881, installPath=/home/admin/oceanbase, runPath=/home/admin/oceanbase, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=4, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:45:48.993570+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:39.611 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.144:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:39.618 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000028 2024-03-13 15:58:39.620 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000021 2024-03-13 15:58:39.622 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000027 2024-03-13 15:58:39.621 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:39.623 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.623 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.623 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.623 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:58:39.624 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster 2024-03-13 15:58:39.624 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000028 2024-03-13 15:58:39.625 INFO 1433467 --- [pool-Check host task for host58,9a2ca15aada1438f,b7134914dd98] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000021 2024-03-13 15:58:39.626 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 2024-03-13 15:58:39.626 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000002 2024-03-13 15:58:39.627 INFO 1433467 --- [pool-Check host task for host39,9a2ca15aada1438f,b44ab9f90750] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000027 2024-03-13 15:58:39.629 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:58:39.630 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:58:39.633 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000002 2024-03-13 15:58:39.634 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000008 2024-03-13 15:58:39.635 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_monagent, skipPorts=false), params:null 2024-03-13 15:58:39.640 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000008 2024-03-13 15:58:39.641 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:39.641 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.148:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:58:39.647 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000018 2024-03-13 15:58:39.655 WARN 1433467 --- [pool-Check host task for clockdiff30,959ecdb8a2d34e22,b42c140b275f] c.o.ocp.core.task.util.ClockDiff : Clock-diff failed. args=[10.10.10.145], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50) at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95) at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52) at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:326) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.655 ERROR 1433467 --- [pool-Check host task for clockdiff30,959ecdb8a2d34e22,b42c140b275f] c.o.ocp.core.util.ExceptionUtils : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.10.10.145] 2024-03-13 15:58:39.655 WARN 1433467 --- [pool-Check host task for clockdiff30,959ecdb8a2d34e22,b42c140b275f] c.o.o.c.h.manager.HostCheckManagerImpl : Check host clock diff fail. hostId=1, ip=10.10.10.145, exception={} com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.10.10.145 at sun.reflect.GeneratedConstructorAccessor421.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96) at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90) at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:328) at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:315) at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:84) at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-13 15:58:39.656 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000023 2024-03-13 15:58:39.664 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000023 2024-03-13 15:58:39.671 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000029 2024-03-13 15:58:39.674 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.143:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:39.678 INFO 1433467 --- [pool-Check host task for host12,9a2ca15aada1438f,56eb654c32dc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000018 2024-03-13 15:58:39.679 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.142:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:39.681 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000029 2024-03-13 15:58:39.683 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000003 2024-03-13 15:58:39.690 INFO 1433467 --- [pool-Check host task for host5,9a2ca15aada1438f,07bfc92994bc] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000003 2024-03-13 15:58:39.697 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.146:62888/api/v1/process/info, request body:GetProcessInfoRequest(processName=ocp_agentd, skipPorts=false), params:null 2024-03-13 15:58:39.705 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000009 2024-03-13 15:58:39.711 INFO 1433467 --- [pool-Check host task for host33,9a2ca15aada1438f,16a93cbeadb7] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000009 2024-03-13 15:58:39.723 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 4, sqlPort: 2881 got ObProcessInfoResult(pid=1512704, runPath=/home/admin/oceanbase, runUser=admin, execute=/home/admin/oceanbase/bin/observer, installPath=/home/admin/oceanbase) 2024-03-13 15:58:39.723 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000024 2024-03-13 15:58:39.728 INFO 1433467 --- [pool-Check host task for host29,9a2ca15aada1438f,c518bba8afc1] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000024 2024-03-13 15:58:39.731 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:39.732 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.732 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.h.s.HostProcessServiceImpl : Refreshing process running info: processId=1000030 2024-03-13 15:58:39.732 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.148, sql port:2881 2024-03-13 15:58:39.732 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.733 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:58:39.738 INFO 1433467 --- [pool-Check host task for host21,9a2ca15aada1438f,10e07568827e] c.o.o.c.a.p.AbstractProcessOperator : Refreshing process running info: id=1000030 2024-03-13 15:58:39.738 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=2, clusterName=obcluster, obClusterId=1 2024-03-13 15:58:39.738 INFO 1433467 --- [ForkJoinPool-1386605-worker-1,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=2, name=obcluster, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster","ObClusterId":1,"ObRegion":"obcluster","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.148:2882","role":"LEADER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316540069000}, createTime=2023-08-24T15:55:50+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:58:39.826 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:39.827 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.827 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.827 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.827 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SHOW PARAMETERS WHERE scope = 'cluster' AND name LIKE ? , args: [cluster] 2024-03-13 15:58:39.867 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:39.868 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.868 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.868 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.868 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT `zone`, `status`, `region`, `idc` FROM `DBA_OB_ZONES` 2024-03-13 15:58:39.871 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.871 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.872 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT zone, svr_ip, svr_port, sql_port as inner_port, with_rootserver, UPPER(`status`) as `status`, build_version, UNIX_TIMESTAMP(NVL(`stop_time`, 0))*1000000 as stop_time, UNIX_TIMESTAMP(NVL(`start_service_time`, 0))*1000000 as start_service_time, UNIX_TIMESTAMP(NVL(`block_migrate_in_time`, 0))*1000000 as block_migrate_in_time, id FROM DBA_OB_SERVERS ORDER BY zone ASC, id ASC 2024-03-13 15:58:39.875 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:58:39.884 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=2, name=zone3, idcId=2, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:39.884 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:59+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:58:39.895 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=2, ip=10.10.10.147, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=2, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:59:58.632894+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:39.895 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:58:39.902 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=3, name=zone2, idcId=3, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:39.902 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:58:39.911 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=3, ip=10.10.10.146, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=true, hostId=3, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:54:13.038428+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:39.911 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605203] detect zone changed, before=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, current=ObZoneEntity{id=null, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null} 2024-03-13 15:58:39.923 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] zone updated, zone=ObZoneEntity{id=1, name=zone1, idcId=1, status=RUNNING, innerStatus=ACTIVE, compactionStatus=null, startTime=null, stopTime=null}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:39.923 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605303] detect server changed, before=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15+08:00, stopTime=1970-01-01T08:00+08:00}, current=ObServerEntity{id=null, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=null, runPath=null, runUser=null, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00} 2024-03-13 15:58:39.934 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] server updated, server=ObServerEntity{id=1, ip=10.10.10.145, port=2882, sqlPort=2881, installPath=/home/admin/observer, runPath=/home/admin/observer, runUser=admin, version=4.2.1.4_104000052024022918-3246b00b20dbd0cf1e1bef4b67fd31ff8fe7088f(Feb 29 2024 18:36:40), withRootserver=false, hostId=1, status=RUNNING, innerStatus=ACTIVE, startTime=2024-03-12T17:57:15.202308+08:00, stopTime=1970-01-01T08:00+08:00}, oldStatus=RUNNING, newStatus=RUNNING 2024-03-13 15:58:39.938 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807257,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:39.949 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:39.950 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.950 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:39.950 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:39.951 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = 'min_observer_version' 2024-03-13 15:58:39.955 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : Update cluster attributes info of cluster obcluster1 2024-03-13 15:58:39.958 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 2024-03-13 15:58:39.961 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:58:39.963 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:58:39.982 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.e.internal.template.HttpTemplate : POST request to agent, url:http://10.10.10.145:62888/api/v1/ob/getProcessInfo, request body:ObproxyProcessInfoRequest(sqlPort=2881), params:null 2024-03-13 15:58:40.016 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] .o.o.i.c.ClusterHostOperationServiceImpl : get ob process info host: 1, sqlPort: 2881 got ObProcessInfoResult(pid=1194454, runPath=/home/admin/observer, runUser=admin, execute=/home/admin/observer/bin/observer, installPath=/home/admin/observer) 2024-03-13 15:58:40.038 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.s.o.o.f.ConnectPropertiesBuilder : get credential from obsdk context, userId=200, clusterName=obcluster1, tenantName=sys, dbUser=ocp_monitor 2024-03-13 15:58:40.038 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:40.039 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:10.10.10.145, sql port:2881 2024-03-13 15:58:40.039 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000] 2024-03-13 15:58:40.040 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT arbitration_service_key, arbitration_service, previous_arbitration_service, type FROM DBA_OB_ARBITRATION_SERVICE 2024-03-13 15:58:40.054 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.o.i.c.ClusterSyncServiceImpl : [OCP605] sync cluster done, clusterId=1, clusterName=obcluster1, obClusterId=1 2024-03-13 15:58:40.054 INFO 1433467 --- [ForkJoinPool-1386605-worker-2,,] c.o.o.s.t.b.cluster.SyncAllClustersTask : [OCP605] cluster sync over, doSync=true, cluster=ObClusterOverview(id=1, name=obcluster1, obClusterId=1, vpcId=1, vpcName=system-default, type=PRIMARY, status=RUNNING, syncStatus=VALID, rootserverJson={"ObCluster":"obcluster1","ObClusterId":1,"ObRegion":"obcluster1","ObRegionId":1,"ReadonlyRsList":[],"RsList":[{"address":"10.10.10.145:2882","role":"FOLLOWER","sql_port":2881},{"address":"10.10.10.146:2882","role":"LEADER","sql_port":2881},{"address":"10.10.10.147:2882","role":"FOLLOWER","sql_port":2881}],"Type":"PRIMARY","timestamp":1710316502147000}, createTime=2023-08-24T15:13:24+08:00, creator=admin, obVersion=4.2.1.4, tenantCount=2) 2024-03-13 15:58:40.242 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:40.243 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:41.938 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807280,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:43.243 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:43.244 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:43.938 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807303,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:45.939 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807326,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:46.113 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807328,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:58:46.244 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:46.245 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:46.746 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:58:46.745341+08:00, lastProcessTime=2024-03-13T15:58:36.746029+08:00 2024-03-13 15:58:46.839 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:58:46.745341+08:00 2024-03-13 15:58:46.871 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.m.helper.ExporterRequestHelper : Refresh exporter helper 2024-03-13 15:58:46.886 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=1 2024-03-13 15:58:46.888 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 1 2024-03-13 15:58:46.903 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=7 2024-03-13 15:58:46.905 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 7 2024-03-13 15:58:46.927 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=2 2024-03-13 15:58:46.929 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 2 2024-03-13 15:58:46.946 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=3 2024-03-13 15:58:46.947 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 3 2024-03-13 15:58:46.962 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=6 2024-03-13 15:58:46.963 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 6 2024-03-13 15:58:46.974 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=5 2024-03-13 15:58:46.975 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 5 2024-03-13 15:58:46.993 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.agent.HostAgentServiceImpl : Finding OCP agent: hostId=4 2024-03-13 15:58:46.994 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.c.a.p.HostAgentProcessServiceImpl : Getting all OCP agent processes on host 4 2024-03-13 15:58:47.007 INFO 1433467 --- [pool-ocp-schedules-15,08b4a07857cb41b7,228c9ad92c7a] c.o.o.m.helper.ExporterRequestHelper : Refresh exporter helper finished, count=7 2024-03-13 15:58:47.066 INFO 1433467 --- [pool-ocp-schedules-6,6ccefbed60804185,46f8d840fc7c] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters 2024-03-13 15:58:47.908 INFO 1433467 --- [pool-clean-time-series-data15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:58:47.924 INFO 1433467 --- [pool-print-time-series-interval15-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316727: {1710316620=1394, 1710316635=1394, 1710316650=1394, 1710316665=1394, 1710316680=1394, 1710316695=1394, 1710316710=1394, 1710316725=60}, tsDataMapSize: 1122216, readyDataCacheSize: 8288 2024-03-13 15:58:47.939 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807350,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:47.948 INFO 1433467 --- [pool-clean-time-series-data60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : No deletable timestamps. 2024-03-13 15:58:47.949 INFO 1433467 --- [pool-print-time-series-interval60-1,,] c.o.o.m.a.MonitorTimeSeriesDataProcessor : print tsData 1710316727: {1710316560=32, 1710316620=34, 1710316680=6}, tsDataMapSize: 17768, readyDataCacheSize: 18488 2024-03-13 15:58:49.245 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:49.246 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:49.275 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807365,,] c.o.ocp.alarm.core.task.AlarmDetectTask : alarm detect start 2024-03-13 15:58:49.332 WARN 1433467 --- [pool-scanner-1609811041-1,868fcc750da849dc,15863899dcbc] c.o.o.m.helper.MetricSearchContainer : label is null, return all seriesIds, metricName = node_net_bandwidth_bps, seriesId size = 21 2024-03-13 15:58:49.400 INFO 1433467 --- [alarm-detect-26,42cd46fafff740e9,e369e6001a77] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000914, data is empty. 2024-03-13 15:58:49.404 INFO 1433467 --- [alarm-detect-26,42cd46fafff740e9,e369e6001a77] c.o.o.a.c.d.metrics.SqlMetricsQuery : metric: ob_tenant_exists_expired_xa_trans,sql_alarm_type-trans_stat,trans_state-pending,trans_type-xa, startId: 9000914, data is empty. 2024-03-13 15:58:49.449 INFO 1433467 --- [alarm-detect-4,dcdd93e3b6564ea7,1515fe8e999a] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316695 ~ 1710316719 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316710=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.145, host_name=ShuHeCrmSqlTwo, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone1, svr_ip=10.10.10.145})]}, matched=true, message=null, level=2) 2024-03-13 15:58:49.449 INFO 1433467 --- [alarm-detect-4,dcdd93e3b6564ea7,1515fe8e999a] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316695 ~ 1710316719 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316710=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.147, host_name=ShuHeCrmSqlThree, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone3, svr_ip=10.10.10.147})]}, matched=true, message=null, level=2) 2024-03-13 15:58:49.449 INFO 1433467 --- [alarm-detect-4,dcdd93e3b6564ea7,1515fe8e999a] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316695 ~ 1710316719 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316710=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.144, host_name=ShuHeCrmWebTwo, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.144})]}, matched=true, message=null, level=2) 2024-03-13 15:58:49.449 INFO 1433467 --- [alarm-detect-4,dcdd93e3b6564ea7,1515fe8e999a] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316695 ~ 1710316719 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316710=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.146, host_name=ShuHeCrmSqlOne, ob_cluster_id=1, ob_cluster_name=obcluster1, obzone=zone2, svr_ip=10.10.10.146})]}, matched=true, message=null, level=2) 2024-03-13 15:58:49.449 INFO 1433467 --- [alarm-detect-4,dcdd93e3b6564ea7,1515fe8e999a] c.o.o.alarm.core.detect.MetricDetector : alarmType host_ntp_service_not_exist detect time range 1710316695 ~ 1710316719 matched, alarm-level: 2, evaluate result MultiExpressionEvaluateResult(metricEvaluateResults={ntp_service_available == 0=[ExpressionEvaluateResult(matchValue=0.0, metric=ntp_service_available, match=true, value=0.0, values={1710316710=0.0}, valueCount=1, labels={app=HOST, env_type=host, host=10.10.10.143, host_name=ShuHeCrmWebOne, ob_cluster_id=0, ob_cluster_name=, obzone=, svr_ip=10.10.10.143})]}, matched=true, message=null, level=2) 2024-03-13 15:58:49.940 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807373,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:51.575 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,e79ab096f0c54928,cd6c1f9d4180] c.o.o.s.c.trace.RequestTracingAspect : API: [GET /services?User_ID=alibaba&UID=test&Action=GetObProxyConfig&ObproxyClusterName=ShuHeOBProxy&VersionOnly=true, client=10.10.10.142, traceId=e79ab096f0c54928, method=CommonResult com.oceanbase.ocp.server.common.controllerv22.OcpControllerConfig.getObProxyConfig(String,String,String,String,boolean), args=alibaba,test,,ShuHeOBProxy,true,] 2024-03-13 15:58:51.579 INFO 1433467 --- [http-nio-0.0.0.0-8080-exec-137,e79ab096f0c54928,cd6c1f9d4180] c.o.o.s.c.trace.RequestTracingAspect : API OK: [GET /services client=10.10.10.142, traceId=e79ab096f0c54928, duration=5 ms] 2024-03-13 15:58:51.940 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807396,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:52.247 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:52.248 INFO 1433467 --- [pool-9-thread-1,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:53.284 INFO 1433467 --- [pool-ocp-schedules-17,91309d4ce96e4921,51f9b4a9593e] c.o.o.a.service.TaskRepositoryService : markNoHeartbeatToFailed success, affectRow=0 2024-03-13 15:58:53.291 INFO 1433467 --- [pool-ocp-schedules-17,91309d4ce96e4921,51f9b4a9593e] c.o.o.a.service.TaskRepositoryService : heartbeat done, affectRow=1, taskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:58:53.291 INFO 1433467 --- [pool-ocp-schedules-17,91309d4ce96e4921,51f9b4a9593e] c.o.o.alarm.utils.task.AlarmTaskManager : updateHeartbeat, tryCount=1, heartbeatCount=1, currentTaskUuidList=[c1488e2e-0936-46c6-b03d-d8a2355aff96] 2024-03-13 15:58:53.941 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807419,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:55.248 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.s.RateLimitFacadeServiceImpl : 10.10.10.148:8080 refreshes policies... 2024-03-13 15:58:55.249 INFO 1433467 --- [pool-9-thread-2,,] c.o.o.s.i.r.util.RateLimitCounter : After refresh, size of counterEntries :0 2024-03-13 15:58:55.762 INFO 1433467 --- [pool-ocp-schedules-6,b572fc1aef3d43df,11c4b2bc52ab] c.o.o.monitor.schedule.MonitorSchedules : measurementIdkey sync key id start. 2024-03-13 15:58:55.762 INFO 1433467 --- [pool-ocp-schedules-6,b572fc1aef3d43df,11c4b2bc52ab] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=609459010050985984 2024-03-13 15:58:55.766 INFO 1433467 --- [pool-ocp-schedules-6,b572fc1aef3d43df,11c4b2bc52ab] c.o.o.monitor.schedule.MonitorSchedules : recordIdKey sync key id start. 2024-03-13 15:58:55.767 INFO 1433467 --- [pool-ocp-schedules-6,b572fc1aef3d43df,11c4b2bc52ab] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624101396131614721 2024-03-13 15:58:55.770 INFO 1433467 --- [pool-ocp-schedules-6,b572fc1aef3d43df,11c4b2bc52ab] c.o.o.monitor.schedule.MonitorSchedules : SeriesIdKey sync key id start. 2024-03-13 15:58:55.770 INFO 1433467 --- [pool-ocp-schedules-6,b572fc1aef3d43df,11c4b2bc52ab] c.o.ocp.monitor.service.IdKeyService : Sync key and id, currentMaxSeriesId=624425259050733570 2024-03-13 15:58:55.941 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807442,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSelfcure, method=observerAlarmEvents 2024-03-13 15:58:56.114 INFO 1433467 --- [SimpleAsyncTaskExecutor-89807445,,] c.o.o.s.common.DistributedLockAspect : Get distributed lock, lockKey=AlarmEventProcessForSnapshot, method=observerAlarmEvents 2024-03-13 15:58:56.749 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process alarm start, currentTimestamp=2024-03-13T15:58:56.748118+08:00, lastProcessTime=2024-03-13T15:58:46.745341+08:00 2024-03-13 15:58:56.795 INFO 1433467 --- [pool-Check obproxy12,dc64683370ae43bc,d07a8ffb1dec] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.143 2883 2024-03-13 15:58:56.796 INFO 1433467 --- [pool-Check obproxy6,dc64683370ae43bc,e75e32b76238] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.142 2883 2024-03-13 15:58:56.796 INFO 1433467 --- [pool-Check obproxy5,dc64683370ae43bc,25958434e19a] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.144 2883 2024-03-13 15:58:56.796 INFO 1433467 --- [pool-Check obproxy15,dc64683370ae43bc,2454a9dd267f] c.o.o.s.o.o.SystemOdpOperatorFactory : connect obproxy 10.10.10.148 2883 2024-03-13 15:58:56.880 INFO 1433467 --- [alarm-task-2,bcfb36a99c1c4269,34bd800e3650] c.o.o.alarm.core.process.AlarmProcessor : process done, currentTimestamp=2024-03-13T15:58:56.748118+08:00 2024-03-13 15:58:57.072 INFO 1433467 --- [pool-ocp-schedules-11,7f718af4368a446a,6b1a904f3cba] c.o.ocp.monitor.OcpMonitorManager : Refresh cached second exporters