OCP 社区版 4.2.2 日志频繁提示 clockdiff 失败问题

现象

ocp-server.log 里大量提示 clockdiff 相关的异常。信息如下。

2024-07-01 11:54:12.906  WARN 6632 --- [pool-host-check-clockdiff3,3792da732bad4012,f82b5b0c5830] c.o.ocp.core.task.util.ClockDiff         : Clock-diff failed. args=[10.0.0.37  ], exception={}. Please try other ocp.host.check.clock-diff.mode in System Parameters

org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
        at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
        at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
        at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153)
        at com.oceanbase.ocp.core.util.LocalCommandExecutor.exec(LocalCommandExecutor.java:50)
        at com.oceanbase.ocp.core.task.util.ClockDiff.execute(ClockDiff.java:95)
        at com.oceanbase.ocp.core.task.util.ClockDiff.diffWithIcmpTimestamp(ClockDiff.java:52)
        at com.oceanbase.ocp.core.task.util.ClockDiff$ClockDiffMode.doRun(ClockDiff.java:172)
        at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:344)
        at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:333)
        at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73)
        at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

2024-07-01 11:54:12.906 ERROR 6632 --- [pool-host-check-clockdiff3,3792da732bad4012,f82b5b0c5830] c.o.ocp.core.util.ExceptionUtils         : Checked Exception: com.oceanbase.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [Execute clock diff failed., 10.0.0.37  ]
2024-07-01 11:54:12.906  WARN 6632 --- [pool-host-check-clockdiff3,3792da732bad4012,f82b5b0c5830] c.o.o.c.h.manager.HostCheckManagerImpl   : Check host clock diff fail. hostId=3, ip=10.0.0.37  , exception={}

com.oceanbase.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=Execute clock diff failed.,10.0.0.37  
        at sun.reflect.GeneratedConstructorAccessor444.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.oceanbase.ocp.core.util.ExceptionUtils.newException(ExceptionUtils.java:96)
        at com.oceanbase.ocp.core.util.ExceptionUtils.throwException(ExceptionUtils.java:90)
        at com.oceanbase.ocp.core.util.ExceptionUtils.unExpected(ExceptionUtils.java:77)
        at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.doCheckHostClockDiff(HostCheckManagerImpl.java:346)
        at com.oceanbase.ocp.compute.host.manager.HostCheckManagerImpl.checkHostClockDiff(HostCheckManagerImpl.java:333)
        at com.oceanbase.ocp.core.task.engine.util.DistributedExclusiveExecutor.lambda$doExecute$0(DistributedExclusiveExecutor.java:73)
        at com.oceanbase.ocp.common.trace.TraceDecorator.lambda$decorate$0(TraceDecorator.java:33)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

但是在 ocp 下测试 clockdiff, 不用sudo 是权限报错,用了 sudo就是正常的。

[admin@RS-OBOPC-P1 log]$ clockdiff 10.0.0.37  
clockdiff: socket: Operation not permitted
[admin@RS-OBOPC-P1 log]$ sudo clockdiff 10.0.0.37  
.
host=10.0.0.37   rtt=750(187)ms/0ms delta=0ms/0ms Mon Jul  1 12:00:08 2024
[admin@RS-OBOPC-P1 log]$ pwd
/home/admin/ocp/log
[admin@RS-OBOPC-P1 log]$ 

OCP 版本号: 4.2.2-20240315150922

用commond -v clockdiff查看命令是否在/usr/bin/下

1 个赞

不在。copy 一个过去,修复了。
谢谢!

[root@RS-OBOPC-P1 ~]# su - admin
Last login: Mon Jul  1 14:57:06 CST 2024 on pts/3
[admin@RS-OBOPC-P1 ~]$ which clockdiff
/usr/sbin/clockdiff
[admin@RS-OBOPC-P1 ~]$ command -v clockdiff
/usr/sbin/clockdiff
[admin@RS-OBOPC-P1 ~]$ sudo cp `which clockdiff` /usr/bin/
[admin@RS-OBOPC-P1 ~]$ command -v clockdiff
/bin/clockdiff
[admin@RS-OBOPC-P1 ~]$