obd启动cluster,proxy总是启动失败

OceanBase
,
#1

【产品名称】OceanBase

【产品版本】3.1.0社区版

【问题描述】[ERROR] obproxy start failed ,cat: /home/admin/oceanbase-ce/run/obproxy-192.168.0.14-2883.pid: No such file or directory

[2021-10-13 10:28:04] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [INFO] Start obproxy

[2021-10-13 10:28:04] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – admin@192.168.0.14 execute: bash -c ‘if [ -f /home/admin/oceanbase-ce/bin/obproxy ]; then exit 1; else exit 0; fi;’

[2021-10-13 10:28:04] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – exited code 1, error output:

[2021-10-13 10:28:04] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – 192.168.0.14 port check

[2021-10-13 10:28:04] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – admin@192.168.0.14 execute: cat /home/admin/oceanbase-ce/run/obproxy-192.168.0.14-2883.pid

[2021-10-13 10:28:04] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – exited code 1, error output:

cat: /home/admin/oceanbase-ce/run/obproxy-192.168.0.14-2883.pid: No such file or directory

[2021-10-13 10:28:04] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – starting 192.168.0.14 obproxy

[2021-10-13 10:28:04] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – admin@192.168.0.14 set env LD_LIBRARY_PATH to ‘/home/admin/oceanbase-ce/lib:’

[2021-10-13 10:28:04] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – admin@192.168.0.14 execute: cd /home/admin/oceanbase-ce; /home/admin/oceanbase-ce/bin/obproxy -o enable_strict_kernel_release=False,enable_cluster_checkout=False --listen_port 2883 --prometheus_listen_port 2884 --rs_list ‘192.168.0.12:2881; 192.168.0.13:2881; 192.168.0.15:2881’ --cluster_name ‘obcluster’

[2021-10-13 10:28:05] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – exited code 0

[2021-10-13 10:28:05] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – admin@192.168.0.14 set env LD_LIBRARY_PATH to ‘’

[2021-10-13 10:28:05] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – admin@192.168.0.14 execute: ps -aux | grep -e ‘/home/admin/oceanbase-ce/bin/obproxy -o enable_strict_kernel_release=False,enable_cluster_checkout=False --listen_port 2883 --prometheus_listen_port 2884 --rs_list 192.168.0.12:2881; 192.168.0.13:2881; 192.168.0.15:2881 --cluster_name obcluster$’ | grep -v grep | awk ‘{print $2}’ > /home/admin/oceanbase-ce/run/obproxy-192.168.0.14-2883.pid

[2021-10-13 10:28:05] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – exited code 0

[2021-10-13 10:28:05] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [INFO] obproxy program health check

[2021-10-13 10:28:08] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – 192.168.0.14 program health check

[2021-10-13 10:28:08] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – admin@192.168.0.14 execute: cat /home/admin/oceanbase-ce/run/obproxy-192.168.0.14-2883.pid

[2021-10-13 10:28:08] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – exited code 0

[2021-10-13 10:28:08] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – admin@192.168.0.14 execute: bash -c ‘cat /proc/net/{tcp,udp}’ | awk -F’ ’ ‘{print $2,$10}’ | grep ‘00000000:0B43’ | awk -F’ ’ ‘{print $2}’ | uniq

[2021-10-13 10:28:09] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] – exited code 0

[2021-10-13 10:28:09] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [WARNING] [WARN] failed to start 192.168.0.14 obproxy

[2021-10-13 10:28:09] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] - sub start ref count to 0

[2021-10-13 10:28:09] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [DEBUG] - export start

[2021-10-13 10:28:09] [128d1b22-2bcd-11ec-b1cc-0202c0a8000f] [ERROR] [ERROR] obproxy start failed

#2

/home/admin/oceanbase-ce/run/

这个目录下没有读写权限吗？

#3

这个目录下面admin具备读写权限

-rw-rw-r-- 1 admin admin 5 Oct 13 10:28 obproxy-192.168.0.14-2883.pid

#4

翻了一下obproxy的log文件,如下

[2021-10-13 10:38:17.230088] INFO [PROXY] obproxymain.cpp:480 [2060][Y0-0] [lt=3] [dc=0] has no inherited sockets, start new obproxy(info={isinherited:false, upgradeversion:-1, needconnaccept:true, userrejected:0, fd:-1, receivedsig:-1, subpid:-1, gracefulexitendtime:0, gracefulexitstarttime:0, activeclientvccount:-1, localaddr:"0.0.0.0", rcstatus:"", hucmd:"", state:"HUSTATEWAITHUCMD", hustatus:"", isparent:true, substatus:"", lastparentstatus:"", lastsubstatus:"", upgradeversionbuf:"", argc:1, argv[0]="./bin/obproxy", inheritedargv[0]="./bin/obproxy", inheritedargv[1]="(null)", inheritedargv[2]="(null)", inheritedargv[3]="(null)"})

[2021-10-13 10:38:17.234983] INFO [PROXY] ob_proxy.cpp:596 [2060][Y0-0] [lt=26] [dc=0] fail to load config from file, but we can get the config from OCP by sql later(ret=0)

[2021-10-13 10:38:17.235062] WARN [PROXY] getkernelreleasebyuname (obconfigserverprocessor.cpp:1039) [2060][Y0-0] [lt=14] [dc=0] unknown uname release(uinfo.release="4.18.0-80.el8.x86_64", ret=-4016)

[2021-10-13 10:38:17.235084] WARN [PROXY] initproxykernelrelease (obconfigserverprocessor.cpp:1152) [2060][Y0-0] [lt=22] [dc=0] fail to getkernelreleasebyuname(ret=-4016)

[2021-10-13 10:38:17.235088] WARN [PROXY] init (obconfigserver_processor.cpp:120) [2060][Y0-0] [lt=4] [dc=0] fail to init proxy kernel release(ret=-4016)

[2021-10-13 10:38:17.235092] ERROR [PROXY] initconnpool (ob_proxy.cpp:545) [2060][Y0-0] [lt=4] [dc=0] fail to init config server processor(ret=-4016) BACKTRACE:0x495929 0x47d3bb 0x420b63 0x44da65 0x44dc60 0x44dfa0 0x42e485 0x432269 0x407f83 0x7fc5e80c8555 0x41ce7f 

[2021-10-13 10:38:17.235301] WARN [PROXY] initconfig (obproxy.cpp:774) [2060][Y0-0] [lt=209] [dc=0] fail to init connection pool(ret=-4016)

[2021-10-13 10:38:17.235306] ERROR [PROXY] init (ob_proxy.cpp:169) [2060][Y0-0] [lt=5] [dc=0] fail to init config(ret=-4016) BACKTRACE:0x495929 0x47d3bb 0x420b63 0x44e2d7 0x42e485 0x432269 0x407f83 0x7fc5e80c8555 0x41ce7f 

[2021-10-13 10:38:17.235318] WARN [PROXY] init (ob_proxy.cpp:226) [2060][Y0-0] [lt=12] [dc=0] fail to init obproxy(ret=-4016)

[2021-10-13 10:38:17.235322] ERROR [PROXY] dostartwork (obproxymain.cpp:661) [2060][Y0-0] [lt=3] [dc=0] obproxy init failed(ret=-4016) BACKTRACE:0x495929 0x47d3bb 0x420b63 0x42e545 0x432269 0x407f83 0x7fc5e80c8555 0x41ce7f 

----------------------------------------------------------------------------------

[2021-10-13 10:38:17.235335] ERROR [PROXY] start (obproxymain.cpp:483) [2060][Y0-0] [lt=13] [dc=0] fail to do start work(ret=-4016) BACKTRACE:0x495929 0x47d3bb 0x420b63 0x43257f 0x407f83 0x7fc5e80c8555 0x41ce7f 

#5

[2021-10-13 10:38:17.234983] INFO [PROXY] ob_proxy.cpp:596 [2060][Y0-0] [lt=26] [dc=0] fail to load config from file, but we can get the config from OCP by sql later(ret=0)


为什么找不到配置文件？

你这里的obproxy肯定没有部署OCP的，所以肯定不能从OCP中拉取配置。要看下为什么obproxy 无法读取配置文件。

#6

obproxy环境变量路径是否要配置？

例如export LD_LIBRARY_PATH


1 个赞
#7

目前报错看日志与启动命令是不相符的。能否提供etc/obproxy_config.bin和更多的日志文件

#8

重新顶一下，这个etc/obproxy_config.bin日志文件都是空的，没有生成，日志如下

[2021-11-03 14:29:09] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] - sub display ref count to 0

[2021-11-03 14:29:09] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] - export display

[2021-11-03 14:29:09] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] - Call obproxy-pyscriptstart-3.1.0 for obproxy-3.1.0-0b17cf0459a3b53c5a2febb6572894d183154c64

[2021-11-03 14:29:09] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] - import start

[2021-11-03 14:29:09] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] - add start ref count to 1

[2021-11-03 14:29:09] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [INFO] Start obproxy

[2021-11-03 14:29:09] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- admin@192.168.63.4 execute: bash -c 'if [ -f /home/admin/oceanbase-ce/bin/obproxy ]; then exit 1; else exit 0; fi;' 

[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- exited code 1, error output:


[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- 192.168.63.4 port check

[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- admin@192.168.63.4 execute: cat /home/admin/oceanbase-ce/run/obproxy-192.168.63.4-2883.pid 

[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- exited code 1, error output:

cat: /home/admin/oceanbase-ce/run/obproxy-192.168.63.4-2883.pid: No such file or directory


[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- starting 192.168.63.4 obproxy

[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- admin@192.168.63.4 set env LDLIBRARYPATH to '/home/admin/oceanbase-ce/lib:'

[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- admin@192.168.63.4 execute: cd /home/admin/oceanbase-ce; /home/admin/oceanbase-ce/bin/obproxy -o enablestrictkernelrelease=False,enableclustercheckout=False --listenport 2883 --prometheuslistenport 2884 --rslist '192.168.0.12:2881; 192.168.0.13:2881; 192.168.0.15:2881' --clustername 'obcluster' 

[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- exited code 0

[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- admin@192.168.63.4 set env LDLIBRARYPATH to ''

[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- admin@192.168.63.4 execute: ps -aux | grep -e '/home/admin/oceanbase-ce/bin/obproxy -o enablestrictkernelrelease=False,enableclustercheckout=False --listenport 2883 --prometheuslistenport 2884 --rslist 192.168.0.12:2881; 192.168.0.13:2881; 192.168.0.15:2881 --clustername obcluster$' | grep -v grep | awk '{print $2}' > /home/admin/oceanbase-ce/run/obproxy-192.168.63.4-2883.pid 

[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- exited code 0

[2021-11-03 14:29:10] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [INFO] obproxy program health check

[2021-11-03 14:29:13] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- 192.168.63.4 program health check

[2021-11-03 14:29:13] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- admin@192.168.63.4 execute: cat /home/admin/oceanbase-ce/run/obproxy-192.168.63.4-2883.pid 

[2021-11-03 14:29:13] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] -- exited code 0

[2021-11-03 14:29:13] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [WARNING] [WARN] failed to start 192.168.63.4 obproxy

[2021-11-03 14:29:13] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] - sub start ref count to 0

[2021-11-03 14:29:13] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [DEBUG] - export start

[2021-11-03 14:29:13] [3cd00a72-3c6f-11ec-b621-0202c0a83f01] [ERROR] [ERROR] obproxy start failed