What is ELK?
ELK is the group of three open source projects in Linux. Elastic Search, Logstash and Kibana respectively. Before going into depth let’s have a short definition about these:
Elasticsearch:
This is an open source distribution, reliable, scalable, easy to use and flexible Lucene library based search engine. It provides multitenant-capable text with an HTTP web interface.
Logstash:
It is an open source tool used to store data, collect information, and store it for further use. Kibana is used to retrieve the logs stored by Logstash.
Kibana:
Let’s you transform your data into your own format or specific shape like charts and graphs in Elasticsearch.
So, in this article we will cover the following :
- How to Install Java on Centos 8
- How to add ELK repository to Centos 8
- How to install and Configure Elasticsearch
- How to instaall and configure Kibana on Centos 8
- How to install and configure Logstash on Centos 8
- How to install other ELK tools(Optional)
Step 1: Install Java on Centos 8
Before installing Elasticsearch we must have java installed on our system as Elasticsearch depends on java. So install it before further proceeding.
How to install Java 11 (OpenJdk 11 on RHEL / Centos 8
Step 2: Add ELK repository to Centos 8
After installing java, add ELK repository to Centos 8 and run the following command as Sudo.
For Elasticsearch 7.x
cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo
[elasticsearch -7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elasticsearch.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticse
enabled=1
autorefresh=1
type=rpm-md
EOF
For Elasticsearch 6.x
cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo
[elasticsearch -6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elasticsearch.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticse
enabled=1
autorefresh=1
type=rpm-md
EOF
For Elasticsearch 5.x
cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo
[elasticsearch -5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elasticsearch.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticse
enabled=1
autorefresh=1
type=rpm-md
EOF
After doing so, import GPG key
sudo rpm –import https://artifacts.elastic.co/GPG-KEY-elasticsearch
Now, clear and update your YUM package index.
sudo yum clean all
sudo yum makecache
Step 3: Install and Configure Elasticsearch
As we have done with Elasticsearch repository and now it’s ready to use. Now make sure to run this command to install Elasticsearch.
sudo yum -y install elasticsearch
Double-check that installation completed successfully.
rpm -qi elasticsearch
Set up the JVM options like memory limits and others according to your own needs. For this edit the following file:
Here we have set up maximum size of total heap space.
/etc/elasticsearch/jvm.options
You can adjust according to your system requirements.
Now, start and enable the Elasticsearch services.
Make sure these are properly working.
Let’s create a test index.
curl -X PUT “http://127.0.0.1:9200/mytest_index”
Step 4: Install / Configure Kibana on Centos 8
From added Elasticsearch repository download and install kibana.
sudo yum -y install kibana
Configure it after installation completed.
sudo vim /etc/kibana/kibana.yml
server.host: “0.0.0.0“
server.name: “kibana.example.com”
elasticsearch.url: “http://localhost:9200“
Set up other settings to your own requirements and start kibana services.
sudo systemctl enable –now kibana
Visit http://ip-address:5601 to open kibana dashboard
If you have firewall service active make sure to allow TCP port 5601.
sudo firewall-cmd –add-port=5601/tcp –permanent
sudo firewall-cmd –reload
Step 4: Install / Configure Logstash on Centos 8
The last step is to install and configure Logstash which will act like a centralized logs server for your client systems and runs an agent like filebeat.
sudo yum -y install logstash
Customize settings under the following directory: /etc/logstash/conf.d/ For further information you can check out Logstash configuration manual.
Step 5: Install other ELK tools – (optional)
Some of these tools help you to work smoothly.
Filebeat:
It makes things simple by following lightweight way to forward and centralized logs and files.
Metricbeat:
Helps you to send and collect metrics from your systems and services, from CPU to memory, Redis to NGINX, and many more. It’s also a lightweight way to access system and services statistics.
Packetbeat:
Packetbeat provides a lightweight way for Network Data to increase performance.
Heartbeat:
Monitors the up time of Services. Helps you to know Availability of services.
Auditbeat:
Useful for auditing the activities and processes on your system by users. The tools we have discussed so far can be installed with the give command one time or you can install individually by this command.
sudo yum install filebeat auditbeat metricbeat packetbeat heartbeat-elastic
These add-on tools help you better experience.To configure any tool you can check official ELK stack documentation.
Hope you are all done! If have any queries regarding this tutorial leave a comment!
