Securing Elasticsearch REST API

Why.?

In my one of the previous ELK posts, we discovered the build in REST API that support out of the box in Elasticsearch. It communicates via 9200/tcp. When it comes security, though, this API is a weak aspect as it is fully open, leaving whole cluster in vulnerable to outsiders. Whoever the party can access the URL can perform any action without being subjected to Authentication & Authorization. In this section of topic, I am going to cover how we can integrate these two major security aspect into Elasticsearch.

How.?

As the figures shows, port tcp 9200 is fully restricted from its communication to outside. It can only work on nodes which are part of the cluster and else blocked. The type of the mechanism which implemented to restrict internal only communication is via Linux in-build firewall – firewallD daemon. This still extends port 9200 to transit over Kiana as well as Logstash as they are also part of the ELK stack.

Extending REST access to outside:

The Security design is being implemented in such a way that yet it is possible of extending REST access to outside if anyone wishes to do so. However, this time they have to pass user authentication plus authorization, which not only enforce check valid user credentials but also a granular access control over what they execute on the REST. This task purely operate on NginX web-server and “lua-scripts”

FirewallD Configuration:

• Across all Elasticsearch/ Logstash/ Kibana nodes, Linux firewall limits port 9200 & 9300 communication to themself. This is accomplished via zone “drop” – which block everything, except the source-address & “source-ports” as configured below.

• Note that below X.X.X.X will have to replace with your all ELK stack nodes IP addresses.

firewall-cmd --zone=drop --add-source=X.X.X.X --permanent

firewall-cmd --zone=drop --add-source-port=9200/tcp --permanen

firewall-cmd --zone=drop --add-source-port=9300/tcp --permanent

• Let one Elasticsearch node expose port 8080/TCP via Nginx proxy which enforces authentication + authorization. Zone “Public” will be configured for 8080 tcp port expose which will allow any IP addresses to be connect with.

firewall-cmd  --zone=public  --add-port=8080/tcp

Nginx Proxy Configuration:

Following Installation works on CentOS 7 system.

• Let install required packages to build nginx

yum install -y openssl-devel perl-ExtUtils-Embed GeoIP-devel pcre-devel zlib zlib-devel gcc make

• Download the nginx source

wget https://openresty.org/download/openresty-1.11.2.4.tar.gz

• Extract the download tar archive

tar -xvf openresty-1.11.2.4.tar.gz

• Configuring the environment

./configure --prefix=/opt/nginx --sbin-path=/usr/sbin/nginx --conf-path=/opt/nginx/config/nginx.conf --pid-path=/opt/nginx/nginx.pid --error-log-path=/opt/nginx/logs/error.log --http-log-path=/opt/nginx/logs/access.log --user=nginx --group=nginx --with-luajit --with-http_auth_request_module 

• Once completed with successful build, now its time to install

gmake && gmake install

• Lets create a systemD unit file at => vim /etc/systemd/system/nginx.service

[Unit]
Description=The NGINX HTTP and reverse proxy server
After=syslog.target network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/opt/nginx/nginx.pid
ExecStartPre=/usr/sbin/nginx -t
ExecStart=/usr/sbin/nginx
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

• Let re-load the systemD database

systemctl daemon-reload

• Edit Nginx the configuration file and have reflect the required instruction => vim /opt/nginx/config/nginx.conf

error_log logs/lua.log notice;
events {
         worker_connections  1024;
 }
 http {
    upstream elasticsearch {
         server 192.168.0.1:9200;
    }
    server {
            listen 8080;
            location / {
                         auth_basic "Protected Elasticsearch";
                         auth_basic_user_file passwords;
                         access_by_lua_file 'authorize.lua';
                         proxy_pass http://elasticsearch;
                         proxy_redirect off;
            }
    }
 }

The above configuration instruct nginx that

• auth_basic_user_file => perform username/password check up for any communication that goes via 192.168.0.1:9200 which is in this example, Elasticserach node IP address
• access_by_lua_file => Even after the successful authentication, every users’ HTTP methods will follow up a privilege escalation against privileges that are defined in the lua script which we will be setting up next.

• Finally, make sure lua script is controlling who’s got which level of HTTP access onto the REST API calls => vim /home/dlogs/nginx/authorize.lua

local restrictions = {
  all  = {
    ["^/$"]                             = { "HEAD" }
  },
 
  bob = {
    ["^/?[^/]*/?[^/]*/_search"]         = { "VIEW", "GET" },
    ["^/?[^/]*/?[^/]*/_msearch"]        = { "GET", "POST" },
    ["^/?[^/]*/?[^/]*/_validate/query"] = { "GET", "POST" }
  },
 
  admin = {
    ["^/?[^/]*/?[^/]*/_bulk"]          = { "GET", "POST" },
    ["^/?[^/]*/?[^/]*/_refresh"]       = { "GET", "POST" },
    ["^/?[^/]*/?[^/]*/?[^/]*/_create"] = { "GET", "POST" },
    ["^/?[^/]*/?[^/]*/?[^/]*/_update"] = { "GET", "POST" },
    ["^/?[^/]*/?[^/]*/?.*"]            = { "GET", "POST", "PUT", "DELETE" },
    ["^/?[^/]*/?[^/]*$"]               = { "GET", "POST", "PUT", "DELETE" },
    ["/_aliases"]                      = { "GET", "POST" }
  }
}
 
 
local role = ngx.var.remote_user
 
 
if restrictions[role] == nil then
  ngx.header.content_type = 'text/plain'
  ngx.status = 403
  ngx.say("403 Forbidden: You don't have access to this resource.")
  return ngx.exit(403)
end
 
 
local uri = ngx.var.uri
 
 
local method = ngx.req.get_method()
 
local allowed  = false
 
for path, methods in pairs(restrictions[role]) do
 
 
  local p = string.match(uri, path)
 
  local m = nil
 
 
  for _, _method in pairs(methods) do
    m = m and m or string.match(method, _method)
  end
 
  if p and m then
    allowed = true
  end
end
 
if not allowed then
  ngx.header.content_type = 'text/plain'
  ngx.log(ngx.WARN, "Role ["..role.."] not allowed to access the resource ["..method.." "..uri.."]")
  ngx.status = 403
  ngx.say("403 Forbidden: You don't have access to this resource.")
  return ngx.exit(403)
end

• Start the Nginx service

systemctl start nginx

Now, you need to change Rest API url which, for example, were being;

curl -XGET http://192.168.0.1:9200/testindex/_search 

then the new url and its parameter would be;

curl -XGET --user bob:password http://192.168.0.1:8080/testindex/_search

“I hope this has been very informative for you”