oceanbase监控数据采集问题

【 使用环境 】生产环境
【 OB or 其他组件 】ocp_monagent
【 使用版本 】ob版本:4.2.2.0 / 4.3.5.3
【问题描述】4.2.2.0的ob是使用obd安装的,使用接口调用数据curl -s http://127.0.0.1:8088/metrics/node/ob只有如下数据
curl -s http://127.0.0.1:8088/metrics/node/ob

oceanbase_connectivity{target=“oceanbase”} 1
process_exists{env_type=“host”,name=“observer”} 1
导致grafana中监控数据不全。

4.3.5.3版本的ob使用ocp白屏安装,监控数据正常,接口数据如下
curl http://***:62889/metrics/node/ob

ob_log_level_count{app=“OB”,level=“INFO”,logSourceType=“election”,ob_cluster_id=“1”,ob_cluster_name=“",obzone=“zone1”,svr_ip="”} 3.726885e+06 1783933653490

ob_log_level_count{app=“OB”,level=“INFO”,logSourceType=“observer”,ob_cluster_id=“1”,ob_cluster_name=“",obzone=“zone1”,svr_ip="”} 3.51839502e+08 1783933653543

ob_log_level_count{app=“OB”,level=“INFO”,logSourceType=“rootservice”,ob_cluster_id=“1”,ob_cluster_name=“",obzone=“zone1”,svr_ip="”} 3.2883936e+07 1783933653598

ob_log_level_count{app=“OB”,level=“WDIAG”,logSourceType=“observer”,ob_cluster_id=“1”,ob_cluster_name=“",obzone=“zone1”,svr_ip="”} 3.7045765e+07 1783933653543

ob_log_level_count{app=“OB”,level=“WDIAG”,logSourceType=“rootservice”,ob_cluster_id=“1”,ob_cluster_name=“",obzone=“zone1”,svr_ip="”} 896663 1783933652084

observer_ulimit_max_fd_count{app=“OB”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”} 655350 1783933637493

oceanbase_connectivity{app=“OB”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”,target=“monitordb”} 1 1783933627631

oceanbase_connectivity{app=“OB”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”,target=“oceanbase”} 1 1783933627631

process_boot_time_seconds{app=“OB”,name=“observer”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”} 1.773824803e+09 1783933629430

process_count{app=“OB”,env_type=“host”,name=“observer”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”} 1 1783933629430

process_cpu_percent{app=“OB”,name=“observer”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”} 673.7448470372121 1783933629430

process_exists{app=“OB”,env_type=“host”,name=“observer”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”} 1 1783933629430

process_fd_count{app=“OB”,name=“observer”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”} 1171 1783933629430

process_memory_rss_bytes{app=“OB”,name=“observer”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”} 3.11465787392e+11 1783933629430

process_thread_count{app=“OB”,name=“observer”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”} 3208 1783933629430

process_uptime_seconds{app=“OB”,name=“observer”,ob_cluster_id=“1”,ob_cluster_name=“",obproxy_cluster="_obproxy”,obproxy_cluster_id=“1”,obregion=“",obzone=“zone1”,svr_ip="”} 1.0108825e+07 1783933629430

9 个赞

没碰到,坐大神回答

1 个赞

没弄过,你这是用Prometheus采集吧

1 个赞

此帖子已被社区举报,现已被临时隐藏。

顶顶

dd

不错,有用的经验又增加了。

在 OB 主机上查看 monagent 监听端口

ss -tlnp | grep ocp_monagent

或查看 agent 配置

grep monitor.http.port ${OCP_AGENT_HOME}/conf/config_properties/common_meta.yaml

observer上访问一下这个信息
curl -s http://127.0.0.1:62889/metrics/node/ob

1 个赞

[root@ocean-102-16 ~]$ curl -s http://127.0.0.1:8088/metrics/node/ob

HELP oceanbase_connectivity monitor collected message

TYPE oceanbase_connectivity gauge

oceanbase_connectivity{target=“oceanbase”} 1

HELP process_exists monitor collected message

TYPE process_exists gauge

process_exists{env_type=“host”,name=“observer”} 1
[root@ocean-102-16 ~]$

这个也没数据
[root@ocean-102-16 ~]$ curl -s http://127.0.0.1:8088/metrics/node/obproxy
404 page not found
[root@ocean-102-16 ~]$ curl -s http://127.0.0.1:8088/metrics/obproxy
404 page not found

分析得很透彻,学到了不少新知识。

mark~~

1 个赞

见解很独特受益匪浅。

1 个赞

这篇文章写得真不错,很有启发性。

4.2.2.0 版本 ob_monagent 未独立部署或未启用。OBD 部署默认不带 monagent,需手动安装 obagent(OCP Express 或独立部署)。升级方案:安装 obagent RPM 包后配置 monagent_bin_port 并重启 observer。

1 个赞

端口不应该是62889么?你修改过端口么? 你查一下看看监听的端口到底是多少
– 在 OB 主机上查看 monagent 监听端口

ss -tlnp | grep ocp_monagent

– 或查看 agent 配置

grep monitor.http.port ${OCP_AGENT_HOME}/conf/config_properties/common_meta.yaml

这两个信息 也查看一下

我是手动安装的monagent,端口就是8088

这个问题让我想起了cluster相关的优化,特别是在ob方面,采用obproxy策略很有效。

对于cluster问题,除了文中提到的方法,还可以考虑ob和obproxy的结合使用。

按照我发的查一下 就知道了