[2024-10-17 01:05:42.046] [2241e9e0-8c40-11ef-8568-000c295e321e] [INFO] Check before start ocp-server
[2024-10-17 01:05:42.051] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – oceanbase version check
[2024-10-17 01:05:42.051] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 execute: sudo -n true
[2024-10-17 01:05:42.215] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – exited code 1, error output:
[2024-10-17 01:05:42.215] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] >>> /etc/sudoers: syntax error near line 124 <<<
[2024-10-17 01:05:42.216] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] sudo: parse error in /etc/sudoers near line 124
[2024-10-17 01:05:42.216] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] sudo: no valid sudoers sources found, quitting
[2024-10-17 01:05:42.216] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] sudo: unable to initialize policy plugin
[2024-10-17 01:05:42.216] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG]
[2024-10-17 01:05:42.216] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 execute: [ id -u == “0” ]
[2024-10-17 01:05:42.471] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – exited code 0
[2024-10-17 01:05:42.472] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 execute: cat /root/ocp/run/ocp-server.pid
[2024-10-17 01:05:42.743] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – exited code 1, error output:
[2024-10-17 01:05:42.743] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] cat: /root/ocp/run/ocp-server.pid: No such file or directory
[2024-10-17 01:05:42.744] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG]
[2024-10-17 01:05:42.744] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – user check
[2024-10-17 01:05:42.744] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – port check
[2024-10-17 01:05:42.744] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 execute: bash -c ‘cat /proc/net/{udp*,tcp*}’ | awk -F’ ’ ‘{if($4==“0A”) print $2,$4,$10}’ | grep ‘:1F90’ | awk -F’ ’ ‘{print $3}’ | uniq
[2024-10-17 01:05:43.019] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – exited code 0
[2024-10-17 01:05:43.020] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – java check
[2024-10-17 01:05:43.020] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 append ‘/root/ocp/jre/bin:’ to PATH
[2024-10-17 01:05:43.020] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 execute: java -version
[2024-10-17 01:05:43.481] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – exited code 0
[2024-10-17 01:05:43.482] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – java version <ssh.SshReturn object at 0x2b4ae41b0eb0>
[2024-10-17 01:05:43.483] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – java_major_version 1.8.0
[2024-10-17 01:05:43.483] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – java_update_version 412
[2024-10-17 01:05:43.483] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – clockdiff check
[2024-10-17 01:05:43.483] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 execute: clockdiff -o 127.0.0.1
[2024-10-17 01:05:43.753] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – exited code 0
[2024-10-17 01:05:43.754] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – memory check
[2024-10-17 01:05:43.755] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 execute: cat /proc/meminfo
[2024-10-17 01:05:43.978] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – exited code 0
[2024-10-17 01:05:43.986] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – disk check
[2024-10-17 01:05:43.989] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 execute: df --block-size=1024
[2024-10-17 01:05:44.203] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – exited code 0
[2024-10-17 01:05:44.204] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /dev, total: 33768304640 avail: 33768304640
[2024-10-17 01:05:44.205] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /dev/shm, total: 33779105792 avail: 33779105792
[2024-10-17 01:05:44.205] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /run, total: 33779105792 avail: 33769209856
[2024-10-17 01:05:44.206] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /sys/fs/cgroup, total: 33779105792 avail: 33779105792
[2024-10-17 01:05:44.206] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /, total: 52710309888 avail: 41274359808
[2024-10-17 01:05:44.206] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /usr, total: 59051704320 avail: 54417219584
[2024-10-17 01:05:44.207] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /boot, total: 2046640128 avail: 1810960384
[2024-10-17 01:05:44.207] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /opt, total: 21003583488 avail: 19866902528
[2024-10-17 01:05:44.208] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /home, total: 359204208640 avail: 340332519424
[2024-10-17 01:05:44.208] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /run/user/0, total: 6755823616 avail: 6755823616
[2024-10-17 01:05:44.209] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 execute: df --block-size=1024 /home/root/logs
[2024-10-17 01:05:44.451] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – exited code 0
[2024-10-17 01:05:44.452] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – get disk info for path /home, total: 359204208640 avail: 340332519424
[2024-10-17 01:05:44.453] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – root@192.168.87.220 execute: ls /root/ocp/.bootstrapped
[2024-10-17 01:05:44.645] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – exited code 2, error output:
[2024-10-17 01:05:44.645] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] ls: cannot access /root/ocp/.bootstrapped: No such file or directory
[2024-10-17 01:05:44.645] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG]
[2024-10-17 01:05:44.646] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - plugin ocp-server-ce-py_script_start_check-4.2.2 result: True
[2024-10-17 01:05:44.647] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - Call ocp-server-ce-py_script_start-4.2.1 for ocp-server-ce-4.3.2-20241012145836.el7-610610e2daf63f6df08af686f9a88b6d8cefcc52
[2024-10-17 01:05:44.647] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - import start
[2024-10-17 01:05:44.664] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - add start ref count to 1
[2024-10-17 01:05:44.665] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – metadb connect check
[2024-10-17 01:05:44.666] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – connect 192.168.87.220 -P2883 -uroot@ocp_meta -p******
[2024-10-17 01:05:45.114] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – meta tenant or monitor tenant connect failed, retrying(9)
[2024-10-17 01:05:46.116] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – connect 192.168.87.220 -P2883 -uroot@ocp_meta -p******
[2024-10-17 01:05:46.160] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – meta tenant or monitor tenant connect failed, retrying(8)
[2024-10-17 01:05:47.163] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – connect 192.168.87.220 -P2883 -uroot@ocp_meta -p******
[2024-10-17 01:05:47.183] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – meta tenant or monitor tenant connect failed, retrying(7)
[2024-10-17 01:05:48.185] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – connect 192.168.87.220 -P2883 -uroot@ocp_meta -p******
[2024-10-17 01:05:48.257] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – meta tenant or monitor tenant connect failed, retrying(6)
[2024-10-17 01:05:49.259] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – connect 192.168.87.220 -P2883 -uroot@ocp_meta -p******
[2024-10-17 01:05:49.280] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – meta tenant or monitor tenant connect failed, retrying(5)
[2024-10-17 01:05:50.282] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – connect 192.168.87.220 -P2883 -uroot@ocp_meta -p******
[2024-10-17 01:05:50.292] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – meta tenant or monitor tenant connect failed, retrying(4)
[2024-10-17 01:05:51.294] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – connect 192.168.87.220 -P2883 -uroot@ocp_meta -p******
[2024-10-17 01:05:51.306] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – meta tenant or monitor tenant connect failed, retrying(3)
[2024-10-17 01:05:52.308] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – connect 192.168.87.220 -P2883 -uroot@ocp_meta -p******
[2024-10-17 01:05:52.331] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – meta tenant or monitor tenant connect failed, retrying(2)
[2024-10-17 01:05:53.334] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – connect 192.168.87.220 -P2883 -uroot@ocp_meta -p******
[2024-10-17 01:05:53.346] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – meta tenant or monitor tenant connect failed, retrying(1)
[2024-10-17 01:05:54.348] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – connect 192.168.87.220 -P2883 -uroot@ocp_meta -p******
[2024-10-17 01:05:54.357] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] – meta tenant or monitor tenant connect failed, retrying(0)
[2024-10-17 01:05:54.357] [2241e9e0-8c40-11ef-8568-000c295e321e] [ERROR] meta tenant or monitor tenant connect failed
[2024-10-17 01:05:54.357] [2241e9e0-8c40-11ef-8568-000c295e321e] [ERROR] start ocp-server-ce failed
[2024-10-17 01:05:54.358] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - sub start ref count to 0
[2024-10-17 01:05:54.358] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - export start
[2024-10-17 01:05:54.358] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - plugin ocp-server-ce-py_script_start-4.2.1 result: False
[2024-10-17 01:05:54.358] [2241e9e0-8c40-11ef-8568-000c295e321e] [ERROR] ocp-server-ce start failed
[2024-10-17 01:05:54.359] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - Searching drop_tenant plugin for components …
[2024-10-17 01:05:54.360] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - Searching drop_tenant plugin for oceanbase-ce-4.2.1.8-108000022024072217.el7-499b676f2ede5a16e0c07b2b15991d1160d972e8
[2024-10-17 01:05:54.362] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - Found for oceanbase-ce-py_script_drop_tenant-4.0.0.0 for oceanbase-ce-4.2.1.8
[2024-10-17 01:05:54.363] [2241e9e0-8c40-11ef-8568-000c295e321e] [DEBUG] - connect 192.168.87.221 -P2881 -uroot@sys -p******
1 个赞
yaml文件配置能提供一份么。
1 个赞
在 ~/.obd/cluster/xxx/下
1 个赞
username: root
password: root
port: 22
oceanbase-ce:
version: 4.2.1.8
release: 108000022024072217.el7
package_hash: 499b676f2ede5a16e0c07b2b15991d1160d972e8
192.168.87.221:
zone: zone1
192.168.87.222:
zone: zone2
192.168.87.223:
zone: zone3
servers:
- 192.168.87.221
- 192.168.87.222
- 192.168.87.223
global:
appname: myoceanbase
root_password: aaAA11++
mysql_port: 2881
rpc_port: 2882
home_path: /root/oceanbase
data_dir: /data/1
redo_dir: /data/log1
datafile_size: 4GB
log_disk_size: 8GB
memory_limit: 28GB
ocp_meta_tenant:
tenant_name: ocp_meta
max_cpu: 2.0
memory_size: 4G
ocp_meta_username: root
ocp_meta_password: aaAA11++
ocp_meta_db: meta_database
ocp_monitor_tenant:
tenant_name: ocp_monitor
max_cpu: 2.0
memory_size: 8G
ocp_monitor_username: root
ocp_monitor_password: aaAA11++
ocp_monitor_db: monitor_database
cluster_id: 1729140802
proxyro_password: eEN97rZ5BS
ocp_root_password: ftJI0GWnLE
ocp_meta_tenant_log_disk_size: 6G
enable_syslog_wf: false
max_syslog_file_count: 4
system_memory: 6G
cpu_count: 16
obproxy-ce:
version: 4.3.1.0
package_hash: 835f4803c1f4da186439323b66c51db4662678a3
release: 4.el7
servers: - 192.168.87.220
global:
home_path: /root/obproxy
prometheus_listen_port: 2884
listen_port: 2883
enable_obproxy_rpc_service: false
obproxy_sys_password: jgAPO4kNVL
skip_proxy_sys_private_check: true
enable_strict_kernel_release: false
enable_cluster_checkout: false
depends: - oceanbase-ce
192.168.87.220:
proxy_id: 3860
client_session_id_version: 2
ocp-server-ce:
version: 4.3.2
package_hash: 610610e2daf63f6df08af686f9a88b6d8cefcc52
release: 20241012145836.el7
servers: - 192.168.87.220
global:
home_path: /root/ocp
soft_dir: /home/root/software
log_dir: /home/root/logs
ocp_site_url: http://192.168.87.220:8080
port: 8080
admin_password: aaAA11++
memory_size: 4G
manage_info:
machine: 10
depends: - oceanbase-ce
- obproxy-ce
1 个赞
你是打算220做proxy节点，221，222，223做ob集群？
datafile_size: 200GB
log_disk_size: （memory_limit的3-4倍大小）
这俩太小了，并且确保你每个机器至少有30G内存
- 确保ssh免密登录配置4个节点
2.meta租户连接失败，测试使用root@sys#myoceanbase -P2883去连接下集群排查下是否为2883端口连接失败导致。
3.提供一下启动集群期间的ob日志
1 个赞
Get local repositories ok
Search plugins ok
Load cluster param plugin ok
Open ssh connection ok
Check before start observer ok
Check before start obproxy ok
Check before start ocp-server ok
Start observer ok
observer program health check ok
obshell program health check ok
Connect to observer 192.168.87.221:2881 ok
Start obproxy ok
obproxy program health check ok
Connect to obproxy ok
Initialize obproxy-ce ok
[ERROR] meta tenant or monitor tenant connect failed
[ERROR] start ocp-server-ce failed
[ERROR] ocp-server-ce start failed
Wait for observer init ok
±-------------------------------------------------+
| oceanbase-ce |
±---------------±--------±-----±------±-------+
| ip | version | port | zone | status |
±---------------±--------±-----±------±-------+
| 192.168.87.221 | 4.2.1.8 | 2881 | zone1 | ACTIVE |
| 192.168.87.222 | 4.2.1.8 | 2881 | zone2 | ACTIVE |
| 192.168.87.223 | 4.2.1.8 | 2881 | zone3 | ACTIVE |
±---------------±--------±-----±------±-------+
obclient -h192.168.87.221 -P2881 -uroot -p’aaAA11++’ -Doceanbase -A
cluster unique id: ba54c322-8a5f-5f77-98d3-3f6c8a61fe27-19298dcdf4c-08010204
±-------------------------------------------------+
| obproxy-ce |
±---------------±-----±----------------±-------+
| ip | port | prometheus_port | status |
±---------------±-----±----------------±-------+
| 192.168.87.220 | 2883 | 2884 | active |
±---------------±-----±----------------±-------+
obclient -h192.168.87.220 -P2883 -uroot@proxysys -p’jgAPO4kNVL’ -Doceanbase -A
See https://www.oceanbase.com/product/ob-deployer/error-codes .
Trace ID: a0c9a8d0-8c58-11ef-a38c-000c295e321e
If you want to view detailed obd logs, please run: obd display-trace a0c9a8d0-8c58-11ef-a38c-000c295e321e
You have new mail in /var/spool/mail/root
1 个赞
先按照我上面回的 排查一下
1 个赞
你的磁盘空间不足，调小点试试
1 个赞
只有20能过，怎么回事
1 个赞
df -h看一下
1 个赞
主机
zone1
zong2
zone3
磁盘路径都设置到home目录下了么
麻烦把obd详细日志再发一份
你这个日志是18日早上凌晨2点的，有刚刚部署时候报错的日志么
