Monitoring in Practice¶

OpsAny basic monitoring is a management platform built around Zabbix. Integration with Zabbix gives you its powerful data collection and template management, while OpsAny fills in what Zabbix lacks: CMDB integration, onboarding integration, and visualization. Although we minimize dependence on the Zabbix UI, you still need to be familiar with Zabbix to get the most from basic monitoring.

Integrating Zabbix¶

You integrate Zabbix over its API on the "Collection Management" page of the 【Control Platform】. Only Zabbix LTS releases are supported: 5.0 LTS, 6.0 LTS, and 7.0 LTS. LTS releases are recommended for production.

OpsAny can connect multiple Zabbix Servers and manage them in groups. Since v2.2.2 it deploys Zabbix 7.0.3 by default and integrates it into the Control Platform automatically.

Deploying Zabbix Server¶

OpsAny provides zabbix-install.sh to deploy Zabbix Server with Docker in one step. You can also connect to the Zabbix your organization already runs, or deploy it yourself.

Deploying Zabbix Agent¶

When onboarding hosts in the Control Platform you can select the Zabbix monitoring plugin; OpsAny then deploys Zabbix Agent on the target host automatically (passive mode). Choose the correct monitoring template to collect metrics properly.

Zabbix Agent modes:

Passive mode (default): Zabbix Server polls port 10050 on the Zabbix Agent for data. Port 10051 on the Server and port 10050 on the Agent must be able to reach each other.
Active mode: the Zabbix Agent connects to port 10051 on the Server to send metrics. The HostName field in the Agent configuration must match the configured name, and you must select the corresponding Active template.

If monitoring status looks wrong, sign in to the Zabbix web UI to check. If the Zabbix web UI also looks wrong, investigate the network, templates, and related settings.

The Control Platform and Zabbix¶

Zabbix host tags.

Every monitoring object in Zabbix is a 【Host】, while OpsAny distinguishes hosts, databases, network devices, and so on. To tell hosts from network devices, the Control Platform adds tags to the Host during onboarding.

Host: host_type: server
Network device: host_type: network, device_type: ROUTER|FIREWALL|SWITCH. The latter is taken automatically from the network device model group on the Resource Platform, and custom models are supported.

If you need to sync Zabbix hosts back into the OpsAny Control Platform and network devices are involved, set the tags manually beforehand so the sync succeeds.

How does a Control Platform host map to Zabbix?

A Control Platform host maps to the Zabbix Host Name through its unique identifier. If Zabbix was in use before OpsAny was deployed, we recommend syncing hosts from Zabbix into the Control Platform. The Hostname becomes the unique identifier after sync, and the visible name becomes the name after sync.

Deploying Zabbix Agent¶

When adding a host in the OpsAny 【Control Platform】 you can select the Zabbix monitoring plugin, and OpsAny runs the Zabbix Agent installation script automatically over SSH or the Agent according to the onboarding information. With so many operating systems, we cannot support them all. For unsupported systems, or when automatic installation fails, install manually.

Download the package for your operating system:
https://repo.zabbix.com/zabbix/7.0/
Install manually (details omitted)

Follow the installation steps for your operating system.

Edit the configuration file

vim /etc/zabbix/zabbix_agent2.conf
Server=10.0.0.3           # Change to the Zabbix Server address
ServerActive=10.0.0.3     # Change to the Zabbix Server address
Hostname=opsany-demo      # Change to the host's unique identifier

Start Zabbix Agent

systemctl restart zabbix-agent2
systemctl enable zabbix-agent2

Grafana visualization¶

OpsAny basic monitoring visualizes data with Grafana, a popular open-source data visualization tool. Grafana supports many data sources out of the box and displays Zabbix data through a plugin. Basic monitoring already embeds Grafana so you can build custom dashboards.

Integrating Prometheus¶

OpsAny supports Prometheus integration; the Community Edition lets you install Exporters manually for monitoring. OpsAny uses Prometheus to monitor hosts, components, and applications, and supports custom metric collection plugins under 【Control Platform】 → 【Collection Plugins】 → 【Metric Collection】.

Where to integrate: 【Control Platform】 → 【Collection Management】 → 【Prometheus Integration】, add a Prometheus Server.

Monitoring MySQL with Prometheus¶

1. Deploy the MySQL Exporter¶

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.13.0/mysqld_exporter-0.13.0.linux-amd64.tar.gz
[root@dev src]# tar zxf mysqld_exporter-0.13.0.linux-amd64.tar.gz
[root@dev src]# mv mysqld_exporter-0.13.0.linux-amd64 /usr/local/
[root@dev src]# ln -s /usr/local/mysqld_exporter-0.13.0.linux-amd64/ /usr/local/mysqld_exporter

2. Configure the systemd service¶

[root@dev src]# vim /usr/lib/systemd/system/mysql_exporter.service
[Unit]
Description=https://prometheus.io

[Service]
Restart=on-failure
ExecStart=/usr/local/mysqld_exporter/mysqld_exporter --config.my-cnf=/usr/local/mysqld_exporter/.my.cnf -web.listen-address=0.0.0.0:9104

[Install]
WantedBy=multi-user.target

3. Configure MySQL¶

Sign in to the monitored MySQL instance

[root@dev ~]# mysql -u root -p

Create and authorize a MySQL monitoring account

MariaDB [(none)]> CREATE USER 'opsany_monitor'@'localhost' IDENTIFIED BY 'OpsAny@2020';
MariaDB [(none)]> GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'opsany_monitor'@'localhost';

Configure the monitoring account

[root@dev ~]# vim /usr/local/mysqld_exporter/.my.cnf
[client]
user=opsany_monitor
password=OpsAny@2020

4. Start the exporter¶

systemctl daemon-reload
systemctl restart mysql_exporter.service

5. Test it¶

curl http://127.0.0.1:9104/metrics

If the test looks good, edit the Prometheus configuration to add the target.

Monitoring Redis with Prometheus¶

1. Deploy the Redis Exporter¶

Download and deploy the Redis Exporter

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://github.com/oliver006/redis_exporter/releases/download/v1.25.0/redis_exporter-v1.25.0.linux-amd64.tar.gz
[root@dev src]# tar zxf redis_exporter-v1.25.0.linux-amd64.tar.gz
[root@dev src]# mv redis_exporter-v1.25.0.linux-amd64 /usr/local/
[root@dev src]# ln -s /usr/local/redis_exporter-v1.25.0.linux-amd64/ /usr/local/redis_exporter

Startup parameters

Startup parameters

-redis.addr string: address of the Redis instance; one or more, comma-separated. Defaults to "redis://localhost:6379"
-redis.password string: password of the Redis instance
-web.listen-address string: address the service listens on. Defaults to 0.0.0.0:9121

2. Start the Redis Exporter¶

Create the prometheus runtime user

[root@dev ~]# groupadd prometheus
[root@dev ~]# useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
[root@dev ~]# chown -R prometheus:prometheus /usr/local/redis_exporter

Create the systemd unit

[root@dev ~]# cat > /usr/lib/systemd/system/redis_exporter.service  <<EOF
[Unit]
Description=redis_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/redis_exporter/redis_exporter -redis.addr 127.0.0.1:6379  -redis.password 123456 -web.listen-address 0.0.0.0:9121
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

3. Start the Redis Exporter¶

[root@dev ~]# systemctl start redis_exporter
[root@dev ~]# systemctl enable redis_exporter

Test access

curl http://127.0.0.1:9121/metrics

Monitoring MongoDB with Prometheus¶

Official site: https://github.com/percona/mongodb_exporter

1. Deploy the MongoDB Exporter¶

Download and deploy the MongoDB Exporter

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://github.com/percona/mongodb_exporter/releases/download/v0.34.0/mongodb_exporter-0.34.0.linux-amd64.tar.gz
[root@dev src]# tar zxf mongodb_exporter-0.34.0.linux-amd64.tar.gz
[root@dev src]# mv mongodb_exporter-0.34.0.linux-amd64 /usr/local/
[root@dev src]# ln -s /usr/local/mongodb_exporter-0.34.0.linux-amd64/ /usr/local/mongodb_exporter

2. Start the MongoDB Exporter¶

Create the prometheus runtime user

[root@dev ~]# groupadd prometheus
[root@dev ~]# useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
[root@dev ~]# chown -R prometheus:prometheus /usr/local/mongodb_exporter

Create and authorize a MongoDB monitoring user

use admin
db.createUser({
    user: "prometheus",
    pwd: "prometheus",
    roles: [
        { role: "read", db: "admin" },
        { role: "readAnyDatabase", db: "admin" },
        { role: "clusterMonitor", db: "admin" }
    ]
});

Create the systemd unit

[root@dev ~]# cat > /usr/lib/systemd/system/mongodb_exporter.service  <<EOF
[Unit]
Description=mongodb_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/mongodb_exporter/mongodb_exporter --mongodb.uri=mongodb://prometheus:prometheus@127.0.0.1:27017/admin --collect-all --collector.topmetrics --collector.indexstats --collector.dbstats --web.listen-address=":9216"
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

3. Start the MongoDB Exporter¶

[root@dev ~]# systemctl start mongodb_exporter
[root@dev ~]# systemctl enable mongodb_exporter

Test access

curl http://127.0.0.1:9216/metrics

Monitoring Elasticsearch with Prometheus¶

The Prometheus community provides an Elasticsearch Exporter:

Official site: https://github.com/prometheus-community/elasticsearch_exporter
Downloads: https://github.com/prometheus-community/elasticsearch_exporter/releases

1. Deploy the Elasticsearch Exporter¶

Download and deploy the Elasticsearch Exporter

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://opsany-saas.oss-cn-beijing.aliyuncs.com/exporter/elasticsearch_exporter-1.5.0.linux-amd64.tar.gz
[root@dev src]# tar zxf elasticsearch_exporter-1.5.0.linux-amd64.tar.gz
[root@dev src]# mv elasticsearch_exporter-1.5.0.linux-amd64 /usr/local/
[root@dev src]# ln -s /usr/local/elasticsearch_exporter-1.5.0.linux-amd64/ /usr/local/elasticsearch_exporter

2. Configure the Elasticsearch Exporter¶

Create the prometheus runtime user

[root@dev ~]# groupadd prometheus
[root@dev ~]# useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
[root@dev ~]# chown -R prometheus:prometheus /usr/local/elasticsearch_exporter

Create the systemd unit

[root@dev ~]# cat > /usr/lib/systemd/system/elasticsearch_exporter.service  <<EOF
[Unit]
Description=elasticsearch_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/elasticsearch_exporter/elasticsearch_exporter --web.listen-address 0.0.0.0:9114 --es.uri http://elastic@OpsAny@2023@127.0.0.1:9200 --es.all --es.indices --es.cluster_settings --es.indices_settings --es.shards --es.snapshots
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

3. Start the Elasticsearch Exporter¶

[root@dev ~]# systemctl daemon-reload
[root@dev ~]# systemctl start elasticsearch_exporter
[root@dev ~]# systemctl enable elasticsearch_exporter

Test access

curl http://127.0.0.1:9114/metrics

4. Add a Grafana Dashboard¶

Official dashboard: https://github.com/prometheus-community/elasticsearch_exporter/blob/master/examples/grafana/dashboard.json

Monitoring RabbitMQ with Prometheus¶

There are two ways to monitor RabbitMQ with Prometheus:

RabbitMQ ships an official Prometheus plugin: https://www.rabbitmq.com/prometheus.html
Use a standalone RabbitMQ Exporter

Using RabbitMQ's built-in plugin¶

1. Enable the official plugin¶

# Enable the plugin
rabbitmq-plugins enable rabbitmq_prometheus

# Test access
curl -s localhost:15692/metrics | head -n 3

Using a standalone RabbitMQ Exporter¶

1. Deploy the RabbitMQ Exporter¶

Download and deploy the RabbitMQ Exporter

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://github.com/kbudde/rabbitmq_exporter/releases/download/v1.0.0-RC19/rabbitmq_exporter_1.0.0-RC19_linux_amd64.tar.gz
[root@dev src]# tar xf rabbitmq_exporter_1.0.0-RC19_linux_amd64.tar.gz
[root@dev src]# mv rabbitmq_exporter /usr/local/

2. Configure the RabbitMQ Exporter¶

Create the prometheus runtime user

[root@dev ~]# groupadd prometheus
[root@dev ~]# useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
[root@dev ~]# chown -R prometheus:prometheus /usr/local/rabbitmq_exporter

Create the systemd unit

[root@dev ~]# cat > /usr/lib/systemd/system/rabbitmq_exporter.service  <<EOF
[Unit]
Description=rabbitmq_exporter
After=network.target
[Service]
Type=simple
User=prometheus
Environment="RABBIT_USER=opsany" "RABBIT_PASSWORD=OpsAny@2020" "OUTPUT_FORMAT=JSON" "PUBLISH_PORT=9099" "RABBIT_URL=http://127.0.0.1:15672"
ExecStart=/usr/local/rabbitmq_exporter
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

3. Start the RabbitMQ Exporter¶

[root@dev ~]# systemctl daemon-reload
[root@dev ~]# systemctl start rabbitmq_exporter
[root@dev ~]# systemctl enable rabbitmq_exporter

Test access

curl 127.0.0.1:9099/metrics

Add a Grafana Dashboard¶

You can use the official dashboard

Official dashboard: https://github.com/prometheus-community/elasticsearch_exporter/blob/master/examples/grafana/dashboard.json

Monitoring ZooKeeper with Prometheus¶

1. Deploy the ZooKeeper Exporter¶

Download and deploy the ZooKeeper Exporter

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://github.com/dabealu/zookeeper-exporter/releases/download/v0.1.13/zookeeper-exporter-v0.1.13-linux.tar.gz
[root@dev src]# tar xf zookeeper-exporter-v0.1.13-linux.tar.gz
[root@dev src]# mv zookeeper-exporter-v0.1.13-linux /usr/local/zookeeper-exporter

2. Configure the ZooKeeper Exporter¶

Create the prometheus runtime user

[root@dev ~]# groupadd prometheus
[root@dev ~]# useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
[root@dev ~]# chown -R prometheus:prometheus /usr/local/zookeeper-exporter

Create the systemd unit

[root@dev ~]# cat > /usr/lib/systemd/system/zookeeper_exporter.service  <<EOF
[Unit]
Description=zookeeper_exporter
After=network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target

[Service]
Type=simple
ExecStart=/usr/local/zookeeper-exporter/zookeeper-exporter -zk-hosts 127.0.0.1:2181 -bind-addr :9141
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID

[Install]
WantedBy=multi-user.target
EOF

3. Whitelist the mntr command¶

[root@dev ~]# vim /usr/local/zookeeper/conf/zoo.cfg
Add the following at the end of the file:
4lw.commands.whitelist=*
4lw.commands.whitelist=mntr

Restart ZooKeeper
[root@dev ~]# zkServer.sh stop
[root@dev ~]# zkServer.sh start

4. Start the ZooKeeper Exporter¶

[root@dev ~]# systemctl daemon-reload
[root@dev ~]# systemctl start zookeeper_exporter
[root@dev ~]# systemctl enable zookeeper_exporter

Test access

curl 127.0.0.1:9141/metrics

5. Add a Grafana Dashboard¶

Dashboard JSON download link: https://grafana.com/api/dashboards/11442/revisions/1/download

Grafana ID: 11442

Monitoring Kafka with Prometheus¶

1. Deploy the Kafka Exporter¶

Download and deploy the Kafka Exporter

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://github.com/danielqsj/kafka_exporter/releases/download/v1.7.0/kafka_exporter-1.7.0.linux-amd64.tar.gz
[root@dev src]# tar xf kafka_exporter-1.7.0.linux-amd64.tar.gz
[root@dev src]# mv kafka_exporter-1.7.0.linux-amd64 /usr/local/kafka_exporter

2. Configure the Kafka Exporter¶

Create the prometheus runtime user

[root@dev ~]# groupadd prometheus
[root@dev ~]# useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
[root@dev ~]# chown -R prometheus:prometheus /usr/local/kafka_exporter

Create the systemd unit

[root@dev ~]# cat > /usr/lib/systemd/system/kafka_exporter.service  <<EOF
[Unit]
Description=kafka_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart= /usr/local/kafka_exporter/kafka_exporter --kafka.server=127.0.0.1:9092 --web.listen-address=9308
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

3. Start the Kafka Exporter¶

[root@dev ~]# systemctl daemon-reload
[root@dev ~]# systemctl start kafka_exporter
[root@dev ~]# systemctl enable kafka_exporter

Test access

curl 127.0.0.1:9308/metrics

4. Add a Grafana Dashboard¶

Dashboard JSON download link: https://grafana.com/api/dashboards/7589/revisions/5/download

Grafana ID: 7589

Monitoring Nacos with Prometheus¶

Nacos can be monitored by Prometheus once it exposes its metrics.

Nacos official monitoring documentation: https://nacos.io/en/docs/monitor-guide.html

1. Expose metrics in Nacos¶

Configure application.properties to expose metrics

management.endpoints.web.exposure.include=*

For a cluster, add it on every node.

Test access

Visit 127.0.0.1:8848/nacos/actuator/prometheus to see whether metrics are available

curl 127.0.0.1:8848/nacos/actuator/prometheus

2. Configure Prometheus¶

Edit prometheus.yml to scrape the Nacos metrics

- job_name: 'nacos'
  scrape_interval: 15s
  metrics_path: '/nacos/actuator/prometheus'
  static_configs:
  - targets: ['127.0.0.1:8848']

3. Add a Grafana Dashboard¶

You can use the official dashboard

Official dashboard: https://github.com/nacos-group/nacos-template/blob/master/nacos-grafana.json
Grafana ID: 13221

Monitoring MinIO with Prometheus¶

MinIO exposes Prometheus metrics itself, obtainable in two ways:

Public mode — open access with no authentication
Authenticated mode

Public mode needs no authentication and is therefore insecure; scraping MinIO cluster metrics with authentication is recommended.

MinIO official monitoring documentation: https://min.io/docs/minio/linux/operations/monitoring/collect-minio-metrics-using-prometheus.html

1. Scrape metrics with authentication¶

First set an alias with the MinIO client, then generate the Prometheus configuration

# Run the mc container
docker pull minio/mc
docker run -it --entrypoint=/bin/sh minio/mc

# Set the alias
mc alias set minio http://127.0.0.1:9000 admin admin

# Generate the configuration
mc admin prometheus generate minio

# Configuration output
scrape_configs:
- job_name: minio-job
  bearer_token: eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjQ4MTI0MDA0NTcsImlzcyI6InByb21ldGhldXMiLCJzdWIiOiJhZG1pbiJ9.o2ymp655JwB05XlsXTtZJzDlpYUfT2ITgxOdYyqxiOfnkMznphUALSU0yhgobAaQCLtwAZgnKbjBfwxQJlzKSA
  metrics_path: /minio/v2/metrics/cluster
  scheme: http
  static_configs:
  - targets: ['127.0.0.1:9000']

2. Configure Prometheus¶

Add the generated configuration to Prometheus and restart the service. The Prometheus configuration:

- job_name: minio-job
  bearer_token: eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjQ4MTI0MDA0NTcsImlzcyI6InByb21ldGhldXMiLCJzdWIiOiJhZG1pbiJ9.o2ymp655JwB05XlsXTtZJzDlpYUfT2ITgxOdYyqxiOfnkMznphUALSU0yhgobAaQCLtwAZgnKbjBfwxQJlzKSA
  metrics_path: /minio/v2/metrics/cluster
  scheme: http
  static_configs:
  - targets: ['127.0.0.1:9000']

3. Add a Grafana Dashboard¶

MinIO publishes several official Grafana dashboards, downloadable from Grafana:

Official dashboard: https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/minio-dashboard.json
Grafana ID: 13502

Monitoring Kibana with Prometheus¶

1. Deploy the Kibana Exporter¶

Download and deploy the Kibana Exporter

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://github.com/chamilad/kibana-prometheus-exporter/releases/download/v8.7.x.2/kibana_exporter-v8.7.x.2-linux-amd64
[root@dev src]# mv kibana_exporter-v8.7.x.2-linux-amd64 /usr/local/kibana_exporter

2. Configure the Kibana Exporter¶

Create the prometheus runtime user

[root@dev ~]# groupadd prometheus
[root@dev ~]# useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
[root@dev ~]# chown -R prometheus:prometheus /usr/local/kibana_exporter

Create the systemd unit

[root@dev ~]# cat > /usr/lib/systemd/system/kibana_exporter.service  <<EOF
[Unit]
Description=kibana_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart= /usr/local/kibana_exporter -kibana.uri http://127.0.0.1:5601 -web.listen-address 0.0.0.0:9684
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

Configuration:

- debug: enables verbose output during metric collection, normally used in development.

- kibana.password: the password used for the Kibana API. The Kibana API is used to fetch metrics from Kibana, a data visualization and exploration tool.

- kibana.skip-tls: skips TLS verification for a TLS-secured Kibana URL. TLS is the encryption protocol that protects network communication.

- kibana.uri: the Kibana API URL from which metrics are fetched.

- kibana.username: the username used to authenticate against the Kibana API.

- web.listen-address: the network address the program listens on for HTTP requests. Defaults to ":9684".

- web.telemetry-path: the path on which metrics are exposed over HTTP. Defaults to "/metrics".

3. Start the Kibana Exporter¶

[root@dev ~]# systemctl daemon-reload
[root@dev ~]# systemctl start kibana_exporter
[root@dev ~]# systemctl enable kibana_exporter

Test access

curl http://127.0.0.1:9684/metrics

4. Add a Grafana Dashboard¶

Dashboard JSON download link: https://github.com/chamilad/kibana-prometheus-exporter/blob/master/grafana/dashboard.json

Grafana ID: 21420

Monitoring Logstash with Prometheus¶

1. Deploy the Logstash Exporter¶

Download and deploy the Logstash Exporter

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://github.com/alxrem/prometheus-logstash-exporter/releases/download/0.7.0/prometheus-logstash-exporter_0.7.0_linux_amd64
[root@dev src]# mv prometheus-logstash-exporter_0.7.0_linux_amd64 /usr/local/logstash_exporter

2. Configure the Logstash Exporter¶

Create the prometheus runtime user

[root@dev ~]# groupadd prometheus
[root@dev ~]# useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
[root@dev ~]# chown -R prometheus:prometheus /usr/local/logstash_exporter

Create the systemd unit

[root@dev ~]# cat > /usr/lib/systemd/system/logstash_exporter.service  <<EOF
[Unit]
Description=logstash_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart= /usr/local/logstash_exporter -logstash.host 127.0.0.1  -logstash.port 9600 -web.listen-address 0.0.0.0:9304
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

3. Start the Logstash Exporter¶

[root@dev ~]# systemctl daemon-reload
[root@dev ~]# systemctl start logstash_exporter
[root@dev ~]# systemctl enable logstash_exporter

Test access

curl http://127.0.0.1:9304/metrics

4. Add a Grafana Dashboard¶

Dashboard JSON download link: https://grafana.com/grafana/dashboards/17918-logstash-monitoring/

Grafana ID: 17918

Monitoring PostgreSQL with Prometheus¶

The Prometheus community provides a PostgreSQL Exporter:

Official site: https://github.com/prometheus-community/postgres_exporter
Downloads: https://github.com/prometheus-community/postgres_exporter/releases

1. Deploy the PostgreSQL Exporter¶

Download and deploy the PostgreSQL Exporter

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://github.com/prometheus-community/postgres_exporter/releases/download/v0.15.0/postgres_exporter-0.15.0.linux-amd64.tar.gz
[root@dev src]# tar -zxf postgres_exporter-0.15.0.linux-amd64.tar.gz
[root@dev src]# mv postgres_exporter-0.15.0.linux-amd64 /usr/local/postgres_exporter

2. Configure the PostgreSQL Exporter¶

Create the prometheus runtime user

[root@dev ~]# groupadd prometheus
[root@dev ~]# useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
[root@dev ~]# chown -R prometheus:prometheus /usr/local/postgres_exporter

Create the systemd unit

[root@dev ~]# cat > /usr/lib/systemd/system/postgres_exporter.service  <<EOF
[Unit]
Description=postgres_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/postgres_exporter/postgres_exporter --web.listen-address=":9187"
Environment="DATA_SOURCE_URI=127.0.0.1:5432/postgres?sslmode=disable"
Environment="DATA_SOURCE_USER=postgres"
Environment="DATA_SOURCE_PASS=password"
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

Startup parameters

Environment variables

Environment: sets environment variables. Adjust the database address, username, and password here.

3. Start the PostgreSQL Exporter¶

[root@dev ~]# systemctl daemon-reload
[root@dev ~]# systemctl start postgres_exporter
[root@dev ~]# systemctl enable postgres_exporter

Test access

curl http://127.0.0.1:9187/metrics

4. Add a Grafana Dashboard¶

Dashboard JSON download link: https://grafana.com/grafana/dashboards/9628-postgresql-database/

Grafana ID: 9628

Monitoring SQL Server with Prometheus¶

1. Deploy SQL Server¶

Download and deploy SQL Server

docker pull awaragi/prometheus-mssql-exporter
docker run -e SERVER=127.0.0.1 -e USERNAME=SA -e PASSWORD=password -e DEBUG=app -p 4000:4000 --name prometheus-mssql-exporter awaragi/prometheus-mssql-exporter

Startup parameters:

SERVER: server IP or DNS name (required)
PORT: server port (optional, defaults to 1433)
USERNAME: access user (required)
PASSWORD: access password (required)
ENCRYPT: force encryption (optional, defaults to true)
TRUST_SERVER_CERTIFICATE: sets trustServerCertificate (optional, defaults to true)
DEBUG: comma-separated list of enabled logs (optional; currently supports application and metrics)

Test access:

curl http://127.0.0.1:4000/metrics

2. Add a Grafana Dashboard¶

Dashboard JSON download link: https://grafana.com/grafana/dashboards/13919

Grafana ID: 13919

Monitoring Consul with Prometheus¶

Official Exporter: https://github.com/prometheus/consul_exporter

1. Deploy the Consul Exporter¶

Download and deploy

[root@dev ~]# cd /usr/local/src
[root@dev src]# wget https://github.com/prometheus/consul_exporter/releases/download/v0.12.1/consul_exporter-0.12.1.linux-amd64.tar.gz
[root@dev src]# tar -zxf consul_exporter-0.12.1.linux-amd64.tar.gz
[root@dev src]# mv consul_exporter-0.12.1.linux-amd64.tar.gz /usr/local/consul_exporter

2. Configure the Consul Exporter¶

Create the prometheus runtime user

[root@dev ~]# groupadd prometheus
[root@dev ~]# useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
[root@dev ~]# chown -R prometheus:prometheus /usr/local/consul_exporter

Create the systemd unit

[root@dev ~]# cat > /usr/lib/systemd/system/consul_exporter.service  <<EOF
[Unit]
Description=consul_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/consul_exporter/consul_exporter --consul.server=127.0.0.1:8500 --web.listen-address=":9107"
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

3. Start the Consul Exporter¶

[root@dev ~]# systemctl daemon-reload
[root@dev ~]# systemctl start consul_exporter
[root@dev ~]# systemctl enable consul_exporter

Test access

curl http://127.0.0.1:9107/metrics

4. Add a Grafana Dashboard¶

Dashboard JSON download link: https://grafana.com/grafana/dashboards/12049-consul-exporter-dashboard/

Grafana ID: 12049

Monitoring Nginx with Prometheus¶

1. Enable the stub_status module in Nginx¶

server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;
        location /status {
            stub_status;
            access_log off;
            allow all;
            # allow 10.0.0.0/24;
            # deny all;
        }

        location / {
            root   html;
            index  index.html index.htm;
        }

        #error_page  404              /404.html;

2. Deploy the Nginx Exporter¶

To be completed

3. Check the status.¶

To be completed

Community Edition

Enterprise Edition

Monitoring in Practice¶

Integrating Zabbix¶

Deploying Zabbix Server¶

Deploying Zabbix Agent¶

The Control Platform and Zabbix¶

Deploying Zabbix Agent¶

Grafana visualization¶

Integrating Prometheus¶

Monitoring MySQL with Prometheus¶

1. Deploy the MySQL Exporter¶

2. Configure the systemd service¶

3. Configure MySQL¶

4. Start the exporter¶

5. Test it¶

Monitoring Redis with Prometheus¶

1. Deploy the Redis Exporter¶

2. Start the Redis Exporter¶

3. Start the Redis Exporter¶

Monitoring MongoDB with Prometheus¶

1. Deploy the MongoDB Exporter¶

2. Start the MongoDB Exporter¶

3. Start the MongoDB Exporter¶

Monitoring Elasticsearch with Prometheus¶

1. Deploy the Elasticsearch Exporter¶

2. Configure the Elasticsearch Exporter¶

3. Start the Elasticsearch Exporter¶

4. Add a Grafana Dashboard¶

Monitoring RabbitMQ with Prometheus¶

Using RabbitMQ's built-in plugin¶

1. Enable the official plugin¶

Using a standalone RabbitMQ Exporter¶

1. Deploy the RabbitMQ Exporter¶

2. Configure the RabbitMQ Exporter¶

3. Start the RabbitMQ Exporter¶

Add a Grafana Dashboard¶

Monitoring ZooKeeper with Prometheus¶

1. Deploy the ZooKeeper Exporter¶

2. Configure the ZooKeeper Exporter¶

3. Whitelist the mntr command¶

4. Start the ZooKeeper Exporter¶

5. Add a Grafana Dashboard¶

Monitoring Kafka with Prometheus¶

1. Deploy the Kafka Exporter¶

2. Configure the Kafka Exporter¶

3. Start the Kafka Exporter¶

4. Add a Grafana Dashboard¶

Monitoring Nacos with Prometheus¶

1. Expose metrics in Nacos¶

2. Configure Prometheus¶

3. Add a Grafana Dashboard¶

Monitoring MinIO with Prometheus¶

1. Scrape metrics with authentication¶

2. Configure Prometheus¶

3. Add a Grafana Dashboard¶

Monitoring Kibana with Prometheus¶

1. Deploy the Kibana Exporter¶

2. Configure the Kibana Exporter¶

3. Start the Kibana Exporter¶

4. Add a Grafana Dashboard¶

Monitoring Logstash with Prometheus¶

1. Deploy the Logstash Exporter¶

2. Configure the Logstash Exporter¶

3. Start the Logstash Exporter¶

4. Add a Grafana Dashboard¶

Monitoring PostgreSQL with Prometheus¶

1. Deploy the PostgreSQL Exporter¶

2. Configure the PostgreSQL Exporter¶

3. Start the PostgreSQL Exporter¶

4. Add a Grafana Dashboard¶

Monitoring SQL Server with Prometheus¶

1. Deploy SQL Server¶

2. Add a Grafana Dashboard¶

Monitoring Consul with Prometheus¶

1. Deploy the Consul Exporter¶

2. Configure the Consul Exporter¶

3. Start the Consul Exporter¶

4. Add a Grafana Dashboard¶

Monitoring Nginx with Prometheus¶