How to Install ELK Stack on Ubuntu 18.04

Install ELK Stack on Ubuntu 18.04

In this tutorial we are going to learn how to install ELK Stack on Ubuntu 18.04. We will also install and configure its prerequisites. Elasticsearch, Logstash and Kibana are the three open-source products that are part of the group named ELK stack. It’s a robust solution for research, analyze and visualization of information. As we already know Elasticsearch is distributed, RESTful search and analytics NoSQL engine based on Lucene, the light-weight information processing pipeline or Logstash is employed for handling events and logs from many different sources and Kibana that’s a web application is used for information visualization that works together with Elsticsearch.

I recommend to use a minimal Ubuntu server setup as a basis for the tutorial, that can be a virtual or a root server image with an Ubuntu 18.04 Bionic Beaver minimal install from a web hosting company or you use our minimal server tutorial to install a server from scratch.

Install ELK Stack on Ubuntu 18.04

Step 1. First, ensure your system and apt package lists are fully up-to-date by running the following:

apt-get update -y
apt-get upgrade -y

Step 2. Installing Java.

Elasticsearch requires either OpenJDK or Oracle JDK available on your Ubuntu:

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

Check that java is successfully installed using the following command:

java -version

Step 3. Installing Elasticsearch and Kibana on Ubuntu 18.04 Bionic Beaver.

First, add The Elastic repository:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Next, Create a file at /etc/apt/sources.list.d/elastic.list, and paste the following line into it:

deb https://artifacts.elastic.co/packages/6.x/apt stable main

Install Elasticsearch and Kibana using following command:

sudo apt install elasticsearch kibana

Then, editing the Kibana configuration file:

nano /etc/kibana/kibana.yml

Uncomment the configuration lines for server.port, server.host and elasticsearch.url:

server.host: "localhost"

Restart Kibana and start up Elasticsearch, and both will be ready to go:

sudo systemctl restart kibana
sudo systemctl start elasticsearch

Step 4. Installing Nginx.

Install Nginx and of course to configure it as a reverse proxy to access Kibana from the public IP address:

apt install nginx

Next, create authentication file using the htpasswd command:

echo "idroot:`openssl passwd -apr1 YourPassword`" | sudo tee -a /etc/nginx/htpasswd.kibana

Then, create a new virtual host configuration:

nano /etc/nginx/sites-available

Paste the configuration below:

server {
        listen 80;

        server_name your-site.com;

        auth_basic "Restricted Access";
        auth_basic_user_file /etc/nginx/htpasswd.kibana;

        location / {
            proxy_pass http://localhost:5601;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection 'upgrade';
            proxy_set_header Host $host;
            proxy_cache_bypass $http_upgrade;        
        }
    }

Remove the existing default config, and create a new symlink in sites-enabled for Kibana:

sudo rm /etc/nginx/sites-enabled/default
sudo ln -s /etc/nginx/sites-available/kibana /etc/nginx/sites-enabled/kibana

Start the Nginx service and set it to start automatically on boot:

sudo systemctl enable nginx
sudo systemctl start nginx

Step 6. Installing Logstash.

Install Logsatash and configure it to centralize server logs from clients with filebeat, then filter and transform the Syslog data and move it into the stash:

sudo apt install logstash

Start the Logstash service and set it to start automatically on boot:

systemctl restart logstash
systemctl enable logstash

Step 7. Accessing Kibana.

Open up your browser, and go to the address that you assigned to your Kibana instance in the Nginx configuration. You should be prompted to enter the username and password that you set up for Kibana.

Congratulation’s! You have successfully install and configured Elasticsearch, Logstash, and Kibana on your Ubuntu 18.04 LTS server. Thanks for using this tutorial for installing ELK Stack on Ubuntu 18.04 LTS (Bionic Beaver) system.

How to Install ELK Stack on CentOS 7

Install ELK Stack on CentOS 7

In this tutorial we are going to learn how to install ELK Stack on CentOS 7. We will also install and configure its prerequisites.¬†Elasticsearch, Logstash and Kibana are the three open-source products that are part of the group named ELK stack. It’s a robust solution for research, analyze and visualization of information. As we already know Elasticsearch is distributed, RESTful search and analytics NoSQL engine based on Lucene, the light-weight information processing pipeline or Logstash is employed for handling events and logs from many different sources and Kibana that’s a web application is used for information visualization that works together with Elsticsearch.

I recommend to use a minimal CentOS server setup as a basis for the tutorial, that can be a virtual or a root server image with an CentOS 7 minimal install from a web hosting company or you use our minimal server tutorial to install a server from scratch.

Install ELK Stack on CentOS 7

Step 1. First, ensure your system and apt package lists are fully up-to-date by running the following:

yum -y install epel-release
yum -y update

Step 2. Installing Java.

Elasticsearch is based on Java, make sure you have either OpenJDK or Oracle JDK is installed on your server:

wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http:%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u77-b02/jdk-8u171-linux-x64.rpm"

The next step is the installation with this rpm command:

rpm -ivh jdk-8u171-linux-x64.rpm

Check the installed java version by executing the following command:

java -version

Step 3. Installing Elasticsearch on CentOS 7.

First, import the Elasticsearch public GPG key to the rpm package manager:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Then you should use wget in order to download Elasticsearch and initiate the installation:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.rpm
rpm -ivh elasticsearch-6.2.4.rpm

Now we start the service and will also set to start at boot time:

systemctl enable elasticsearch
systemctl start elasticsearch

Check if Elasticsearch responds to simple requests over HTTP:

curl -X GET http://localhost:9200

Step 4. Installing Kibana.

First, you should use wget in order to download Kibana and then you should install it using the rpm command:

wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.4-x86_64.rpm
rpm -ivh kibana-6.2.4-x86_64.rpm

Next, editing the Kibana configuration file:

nano /etc/kibana/kibana.yml

Uncomment the configuration lines for server.port, server.host and elasticsearch.url:

server.port: 5601
server.host: "localhost"
elasticsearch.url: "http://localhost:9200"

Start the Kibana service and set it to start automatically on boot:

systemctl enable kibana
systemctl start kibana

Step 5. Installing Nginx.

Install Nginx and of course to configure it as a reverse proxy to access Kibana from the public IP address:

yum install nginx httpd-tools

Then, create a new virtual host configuration file in the conf.d directory:

nano /etc/nginx/conf.d/your-domain.com.conf

Paste the configuration below:

server {
    listen 80;
 
    server_name your_domain.com;
 
    auth_basic "Restricted Access";
    auth_basic_user_file /etc/nginx/htpasswd.kibana;
 
    location / {
        proxy_pass http://localhost:5601;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Next, create authentication file using the htpasswd command:

sudo htpasswd -c /etc/nginx/htpasswd.kibana admin
YOUR STRONG PASSWORD

Start the Nginx service and set it to start automatically on boot:

systemctl enable nginx
systemctl start nginx

Step 6. Installing Logstash.

Install Logsatash and configure it to centralize server logs from clients with filebeat, then filter and transform the Syslog data and move it into the stash:

wget https://artifacts.elastic.co/downloads/logstash/logstash-6.2.4.rpm
rpm -ivh logstash-6.2.4.rpm

Start the Logstash service and set it to start automatically on boot:

systemctl restart logstash
systemctl enable logstash

Congratulation’s! You have successfully install ELK Stack on your CentOS 7 server. Thanks for using this tutorial for installing ELK Stack on CentOS 7 system.