Monitor your LEMP server with Netdata

Promise Akpan — Fri, 01 Nov 2019 19:33:35 +0000

Netdata is a highly optimized open-source monitoring agent you install on all your systems: physical and virtual servers, containers, even IoT.
It provides insights in real-time of everything happening on the system it runs. It is designed not to disrupt the core functionalities of systems.
It can also run standalone as well as be integrated into existing monitoring tools like Prometheus, Graphite, Kafka, etc.

In this tutorial, you will learn how to install Netdata as well as configure it to monitor Linux, Nginx, MySQL, and PHP.

Install Netdata

There are several ways to install Netdata specific to your system or environment.
The best and recommended way is to install directly from source using
its automatic one-line installation, which is the default:

bash <(curl -Ss https://my-netdata.io/kickstart.sh)

The automatic installer will install any required system packages and compile Netdata directly on your system.

You can view your Netdata dashboard on your browser by visiting http://SERVER_IP:19999.

Install Nginx

Nginx is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server used to host websites and applications of all sizes. It is a popular alternative to Apache known for its low impact on memory resources, high scalability, and its modular, event-driven architecture which can offer secure, predictable performance.

There are several ways to install Nginx depending on your operating system. You can also build Nginx from source if you require some special functionality not available with packages.

If you require some special functionality not available with packages and ports, you can compile Nginx from source files. However, for this tutorial, Nginx comes bundled with the functionalities you need to let Netdata read metrics from it.

To install the prebuilt Nginx package available for Ubuntu run the following:

apt install nginx

The prebuilt Nginx package for Ubuntu may not get updated to the latest stable version of Nginx.

To install the latest stable version of Nginx on Ubuntu:

Install the prerequisite packages and set up the apt repository for stable Nginx packages:

sudo apt install curl gnupg2 ca-certificates lsb-release

echo "deb http://nginx.org/packages/ubuntu `lsb_release -cs` nginx" \
    | sudo tee /etc/apt/sources.list.d/nginx.list

Next, import an official Nginx signing key so apt could verify the packages authenticity:

curl -fsSL https://nginx.org/keys/nginx_signing.key | sudo apt-key add -

Verify that you have the right key:

sudo apt-key fingerprint ABF5BD827BD9BF62

The output should contain the full fingerprint 573B FD6B 3D8F BC64 1079 A6AB ABF5 BD82 7BD9 BF62 as follows:

pub   rsa2048 2011-08-19 [SC] [expires: 2024-06-14]
      573B FD6B 3D8F BC64 1079  A6AB ABF5 BD82 7BD9 BF62
uid   [ unknown] nginx signing key <signing-key@nginx.com>

Update apt repository and install Nginx with the following commands:

sudo apt update
sudo apt install nginx

Install MySQL

MySQL is an open-source relational database management system.

There are also several ways to install MySQL specific to your operating system.

The MySQL APT repository provides deb packages for installing and managing the MySQL server, client, and other components on Ubuntu.

Run below command to enable the repository.

wget http://repo.mysql.com/mysql-apt-config_0.8.13-1_all.deb
sudo dpkg -i mysql-apt-config_0.8.13-1_all.deb

During the installation of MySQL apt config package, you can select MySQL version (8.0 or 5.7) to install on your system.

Update apt and install MySQL

sudo apt update 
sudo apt install mysql-server

As a security measure, you can further secure your MySQL installation by running this command:

sudo mysql_secure_installation

Install PHP

PHP is an open-source general-purpose scripting language that is especially suited for web development.

Like Nginx, there are several ways to install PHP specific to your operating system.

A quick and easy one is running:

apt install php-fpm php-mysql php-cli

For this tutorial, you will be using the latest stable release of PHP for Ubuntu.

Begin by enabling the Ondrej PHP repository:

sudo apt install software-properties-common
sudo add-apt-repository ppa:ondrej/php

Update apt and install the necessary PHP packages

sudo apt update
sudo apt install php-fpm php-mysql php-cli

You can now edit Nginx server block in the configuration file, so Nginx can process PHP files:

server {

    # . . . your other code

    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/run/php/php7.3-fpm.sock;
    }
}

Restart the Nginx service so that the new configuration takes effect:

sudo systemctl restart nginx

With your LEMP stack ready, let's proceed to configure Netdata to read metrics from Nginx, MySQL, and PHP.

Netdata configuration

When Netdata is installed for the first time or restarted, it autodetects available data sources such as Nginx, PHP, MySQL, Redis, etc.

Now you have to restart the Netdata service so it can auto-detect and collect metrics.

service netdata restart

NOTE: If Netdata isn't collecting metrics after a restart, you probably did not configure your source correctly.

Netdata + Nginx

In order for Netdata to auto-detect an Nginx web server, you need to enable ngx_http_stub_status_module and pass the stub_status directive in the location block of your Nginx configuration file.

Confirm if the required module is enabled or not using the following command:

nginx -V 2>&1 | grep -o with-http_stub_status_module

Add this location block to your Nginx configuration file:


location /stub_status {
    stub_status;
}

You can configure Netdata to monitor many remote Nginx servers other than your local Nginx server by editing the configuration file of Netdata's Nginx module with the following command:

/etc/netdata/edit-config python.d/nginx.conf

Make sure to have the ngx_http_stub_status_module active on any servers you want to monitor.

At the near bottom of the file you will see the below uncommented to serve as a guide for you if you want to add more servers.

localhost:
  name: "local"
  url: "http://localhost/stub_status"

remote1:
  name: "local"
  url: "http://[REMOTE_IP_ADDRESS]/stub_status"

Nginx charts beautifully shown on my Netdata dashboard:

Netdata + MySQL

Like Nginx, Netdata can autodetect MySQL (or its drop-in replacement - MariaDB) if you meet the requirement of an already installed and running python library - MySQLdb.

You also need a local netdata user that connects to the MySQL server on the localhost. This user will be able to gather MySQL statistics without the ability to alter any data or affect MySQL operations.

To create this user, run the following:

create user 'netdata'@'localhost'; 
grant usage on *.* to 'netdata'@'localhost'; 
flush privileges;

Although with the required Python library and netdata user on your MySQL server you can see the charts on your Netdata dashboard, you can monitor other remote MySQL servers that have met the requirements by editing the configuration file of Netdata's MySQL module with the following command:

/etc/netdata/edit-config python.d/mysql.conf

# Other content of the config file appear here
...


remote:
  user     : 'netdata'
  host     : 'example.org'
  port     : 3306 # or any port which MySQL can be accessed

MySQL charts on your dashboard should look like this:

Netdata + PHP

Netdata will autodetect and monitor one or more PHP-FPM instances if php-fpm is enabled with status page and the status page can be accessed via a web server.

Run the following code to view or edit the php-fpm module configuration:

/etc/netdata/edit-config python.d/phpfpm.conf

The file contents:


update_every : 3
priority     : 90100

local:
  url     : 'http://localhost/status'

PHP-FPM charts should appear on your Netdata dashboard like this:

Conclusion

You can set up Netdata to monitor as many services as you want even Hadoop and Kubernetes clusters.

However, do not forget to restart Netdata each time you try to add new services or configurations.

Forem: Promise Akpan