JMXtrans + InfluxDB + Grafana實現Zookeeper性能指標監控
- 2019 年 10 月 22 日
- 筆記
一、總體效果圖
這裡是將集群全部放在一起，可以根據自己的審美看怎麼放
二、監控指標
其中有些指標與第一篇Zookeeper通過四字命令基礎監控（Zabbix）的四字命令的指標是有重複的，二者選一個則可
三、實現
1、influxdb的安裝
1）設置yum源
cat <<EOF | sudo tee /etc/yum.repos.d/influxdb.repo [influxdb] name = InfluxDB Repository - RHEL $releasever baseurl = https://repos.influxdata.com/rhel/$releasever/$basearch/stable enabled = 1 gpgcheck = 1 gpgkey = https://repos.influxdata.com/influxdb.key EOF
2）安裝influxdb
yum install influxdb systemctl start influxdb
3）修改配置文件（元數據以及數據存放目錄）
[root@ip-172-0-0-7 influxDB]# cat /etc/influxdb/influxdb.conf |grep "^s*[^# t].*$" [meta] dir = "/influxDB/meta" [data] dir = "/influxDB/data" wal-dir = "/influxDB/wal" series-id-set-cache-size = 100 [coordinator] [retention] [shard-precreation] [monitor] [http] [logging] [subscriber] [[graphite]] [[collectd]] [[opentsdb]] [[udp]] [continuous_queries] [tls]
4、數據目錄授權
chown -R influxdb.influxdb /influxDB
5、重啟influxdb
systemctl restart influxdb
6、創建zookeeper資料庫，以及創建帳號密碼
CREATE USER "admin" WITH PASSWORD "admin" WITH ALL PRIVILEGES create database zookeeper
2、zookeeper上打開JMX
1）vim ${Zookeeper-Home}/conf/zookeeper-env.sh
JMXLOCALONLY=false JMXDISABLE=false JMXPORT=8999 JMXAUTH=false JMXSSL=false
2、重啟Zookeeper
./zkServer.sh restart
3、JMXtrans的安裝
1）安裝配置jmxtrans
下載地址https://oss.sonatype.org/content/repositories/snapshots/org/jmxtrans/jmxtrans/ 這裡有最新的版本，用老的版本有一些問題 ；JMXtrans可以隨意安裝在任何一台機器上，不是必須安裝在Zookeeper機器上，只要JMXtrans與Zookeeper機器的網路及埠是通的就可以
wget https://oss.sonatype.org/content/repositories/snapshots/org/jmxtrans/jmxtrans/271-SNAPSHOT/jmxtrans-271-20190109.091025-10.rpm yum -y install jmxtrans-271-20190109.091025-10.rpm
2）它的目錄文件
/var/lib/jmxtrans /var/log/jmxtrans /etc/rc.d/init.d/jmxtrans /etc/jmxtrans /usr/share/jmxtrans /usr/share/jmxtrans/bin/jmxtrans /usr/share/jmxtrans/lib/org/jmxtrans /usr/share/jmxtrans/lib/org/jmxtrans/jmxtrans /usr/bin/jmxtrans /run/jmxtrans /run/lock/subsys/jmxtrans
3）編輯zookeeper.json文件並放到/var/lib/jmxtrans/目錄
一般我們的Zookeeper是一個集群，比如當我們的Zookeeper為3台的時候，則需要弄三份zookeeper.json文件，每一份對應Zookeeper集群的每一台機器，通過裡面的host和port對應
{ "servers": [{ "port": 8999, "host": "zookeeper-IP", "alias": "zk", "queries": [{ "outputWriters": [{ "@class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory", "url": "http://localhost:8086/", "database": "zookeeper", "username": "admin", "password": "admin" }], "obj": "org.apache.ZooKeeperService:name0=ReplicatedServer_id*", "attr": ["QuorumSize"] }, { "outputWriters": [{ "@class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory", "url": "http://localhost:8086/", "database": "zookeeper", "username": "admin", "password": "admin" }], "obj": "org.apache.ZooKeeperService:name0=ReplicatedServer_id*,name1=replica*,name2=*", "attr": ["MaxClientCnxnsPerHost", "MaxRequestLatency", "AvgRequestLatency", "MinRequestLatency", "MaxSessionTimeout", "MinSessionTimeout", "OutstandingRequests", "PacketsReceived", "PacketsSent", "PendingRevalidationCount", "TickTime"] }, { "outputWriters": [{ "@class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory", "url": "http://localhost:8086/", "database": "zookeeper", "username": "admin", "password": "admin" }], "obj": "org.apache.ZooKeeperService:name0=ReplicatedServer_id*,name1=replica*,name2=*,name3=InMemoryDataTree", "attr": ["NodeCount", "WatchCount"] }, { "outputWriters": [{ "@class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory", "url": "http://localhost:8086/", "database": "zookeeper", "username": "admin", "password": "admin" }], "obj": "org.apache.ZooKeeperService:name0=ReplicatedServer_id*,name1=replica*,name2=*,name3=Connections,name4=*,name5=*", "resultAlias": "Clients", "attr": ["AvgLatency", "LastLatency", "MaxLatency", "MinLatency", "OutstandingRequests", "PacketsReceived", "PacketsSent"] }, { "outputWriters": [{ "@class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory", "url": "http://localhost:8086/", "database": "zookeeper", "username": "admin", "password": "admin" }], "obj": "org.apache.ZooKeeperService:name0=StandaloneServer_port*", "resultAlias": "Clients", "attr": ["AvgLatency", "LastLatency", "MaxLatency", "MinLatency", "OutstandingRequests", "PacketsReceived", "PacketsSent", "NumAliveConnections"] }, { "outputWriters": [{ "@class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory", "url": "http://localhost:8086/", "database": "zookeeper", "username": "admin", "password": "admin" }], "obj": "java.lang:type=Memory", "resultAlias": "Memory", "attr": ["HeapMemoryUsage", "NonHeapMemoryUsage"] }, { "outputWriters": [{ "@class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory", "url": "http://localhost:8086/", "database": "zookeeper", "username": "admin", "password": "admin" }], "obj": "java.lang:type=OperatingSystem", "resultAlias": "Process", "attr": ["OpenFileDescriptorCount", "ProcessCpuLoad"] }, { "outputWriters": [{ "@class": "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory", "url": "http://localhost:8086/", "database": "zookeeper", "username": "admin", "password": "admin" }], "obj": "java.lang:type=Threading", "resultAlias": "Threading", "attr": ["ThreadCount"] }], "numQueryThreads": 2 }] }
4）啟動JMXtrans
啟動後，查看日誌/var/log/jmxtrans/jmxtrans.log，看是否啟動成功，若有錯誤則根據對應的錯誤解決就可以
/usr/share/jmxtrans/bin/jmxtrans start
4、Grafna上配置數據源
當創建Query，選擇Influxdb數據源後，若有對應的指標出來，表明是OK的
