How to install Prometheus in CentOS 7

H

Prometheus is a server monitoring service. It collects metrics (values) from configured target servers at certain preset intervals, evaluates certain rules and expressions, displays results, and can trigger alerts when certain conditions become true (excessive CPU usage, system run out of memory, run out of space on disk, etc.). Prometheus raises, however, monitoring on a much more complex level than Pinguzo and Netdata.

I must note that although I have used CentOS 7 for both the Prometheus server and the monitoring servers (nodes), the steps below can be followed for installing and configuring Prometheus on any Linux distribution with systemd.

In the tutorial I will not explain each command, because I assume that if you have come to read this article, you should already have some Linux knowledge.

Preparing the system

Prior to installing Prometheus, special care must be taken to set NTP sync – time is a sensitive value in metric analysis. Red Hat 7 (and its derivatives) no longer use the ntpd service, but chronyd – so you should install and configure the chrony package.

If you have the firewall enabled, you must open the following ports:

• 9090 for Prometheus
• 9100 for node_exporter
• 323 for chrony (this if you have not already done it yet)

Depending on the firewall, the commands will be run:

• for firewalld

# firewall-cmd –permanent –add-port=9090/tcp
# firewall-cmd –permanent –add-port=9100/tcp
# firewall-cmd –permanent –add-port=323/udp
# systemctl reload firewall
# firewall-cmd –list-all

• for iptables:

Edit the /etc/sysconfig/iptables file and restart iptables:

# vi /etc/sysconfig/iptables

Add the rules:

-A INPUT -p tcp -m states -state NEW -m tcp –port 9090 -j ACCEPT
-A INPUT -p tcp -m state -state NEW -m tcp –dport 9100 -j ACCEPT

# systemctl restart iptables

Installing the Prometheus server

We will install Prometheus in the directory /opt:

# cd /opt

Navigate to https://prometheus.io/download/ and download the latest version of Prometheus for Linux binaries:

# wget https://github.com/prometheus/prometheus/releases/download/v2.9.2/prometheus-2.9.2.linux-amd64.tar.gz
# tar -zxvf prometheus-2.9.2.linux-amd64.tar.gz
# ln -s prometheus-2.9.2.linux-amd64 prometheus
# useradd prometheus
# chown -R prometheus:prometheus /opt/prometheus-2.9.2.linux-amd64/

Create the Prometheus service
# vi /etc/systemd/system/prometheus.service

Inside the new file we will write the following:

[Unit]
Description=Prometheus Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network-online.target

[Service]
User=root
Restart=on-failure
ExecStart=/opt/prometheus/prometheus –config.file=/opt/prometheus/prometheus.yml

[Install]
WantedBy=multi-user.target

 

Running the Prometheus server

Running commands:

# ln -s /opt/prometheus/prometheus.yml /etc
# systemctl daemon-reload
# systemctl enable –now prometheus
# systemctl status prometheus

View metrics

In the browser, go to http://IP_server_prometheus:9090. Click Graph; enter the expression scrape_duration_seconds and click Execute.

Installing Node Exporter

Node Exporter is installed on each server to be monitored (attention to the port 9100!). We will install Node Exporter in the /opt directory as well; we will download the latest version of Node Exporter (at the time of writing this article, the last stable Node Exporter version is 0.17.0):

# cd / opt
# wget https://github.com/prometheus/node_exporter/releases/download/v0.17.0/node_exporter-0.17.0.linux-amd64.tar.gz
# tar xvfz node_exporter -*.tar.gz
# rm node_exporter-0.17.0.linux-amd64.tar.gz
# ln -s node_exporter-0.17.0.linux-amd64 node_exporter
# chown -R prometheus:prometheus /opt/node_exporter-0.17.0.linux-amd64

Creating the Node Exporter service

We will create a new file in /etc/systemd/system:

# vi /etc/systemd/system/node_exporter.service

In this new file we will write the following (the –no-collector.disktats option has been added because these statistics do not work on virtualized systems – you are free to remove this if you want to):

[Unit]
Description=Node Exporter

[Service]
User=root
ExecStart=/opt/node_exporter/node_exporter –no-collector.diskstats

[Install]
WantedBy=default.target

Starting the service node_exporter:

# systemctl daemon-reload
# systemctl enable –now node_exporter
# systemctl status node_exporter

At this point, browsing the browser at http://IP_server: 9100, you’ll see a page that says Node Exporter and a link called Metrics that contains all metrics for the monitored system (node). Click on Metrics. Now we have to tell Prometheus to read these metrics. On the server where we installed Prometehus, we edit the /opt/prometheus/prometheus.yml file:

# vi /opt/prometheus/prometheus.yml

We add the following lines:

– job_name: ‘node’
static_configs:
– targets:[‘localhost:9100’]

This sequence is added for each server where Node Exporter is installed (localhost is replaced with the corresponding server IP); also job_name: ‘node’ turns into job_name: ‘node1’, etc. – or whatever name you want to name it. Or, the nodes/servers can be added one after the other, as in the example below:

– job_name:’nodes’
static_configs:
– targets:[‘IP_server_01:9100′,’IP_server_02:9100’]

This file is yaml type and is very sensitive to formatting! To verify if it is correctly written, use a validation tool (for example yaml validator).

After editing and saving this file, restart the prometheus service and check its status:

# systemctl restart prometheus
# systemctl status prometheus

The service must be active. If it is not active, the problem might be the configuration file (probably yaml is not properly formatted). To see exactly what’s happening, you can take a look at the /var/log/messages file:

# less /var/log/messages

Once you have added all the nodes where you have installed Node Exporter and want to track them through Prometheus, browse to http://IP_server_prometheus:9090/targets and you will be able to see the status of each server.

Recent Posts

Archives

Categories