In production environments, network security is ineluctable. When elasticsearch is deployed, there are many ways to secure the environment. You can use Ngnix, commercial products like Shield, open source products, or easily selectable plugins via Qbox. However, you can also create your own security plugins and have more control over security. This article is intended to give readers a running start on how to write their own in-house security plugin.

The Elasticsearch network is built using Netty, which gives us the flexibility to add security to the Netty pipeline via plugin. Netty provides an example on how to add the SSL Handler into the pipeline using an example for secure chat. Please refer to this link. Here we will see how to make use of that information to develop an SSL plugin for elasticsearch.

Tutorial

Elasticsearch uses Netty in the transport layer and HTTP layer, so that in our plugin we can extend the classes NettyTransport for the transport layer and NettyHttpServerTransport for the HTTP layer. Let us call these extended classes CustomNettyTransport and CustomNettyHttpServerTransport. If you are new to elasticsearch plugin development, there are several articles available online which will help you through the basics and you can do some experiments before starting with this.

We can look at some basic information on how the network channel works in Netty, which gives an idea about what we are trying to achieve here.

The ChannelPipelineFactory implemented in both layers is used for channel creation whenever a new connection comes. We have to implement the ChannelPipelineFactory for both transport and HTTP in respective CustomNetty classes. The getPipeline method in the ChannelPipelineFactory will return a pipeline which has the encoders, decoders, handlers, and other configurations required for the communication.

For our SSL implementation all weed to add an SSL Handler to this pipeline, but before that we need to configure the handler with required parameters such as protocols, cipher suites, keystore and truststore details.

Creating an SSL Handler

To create an SSL Handler we need the following inputs for the SSL Context:

  • Keystore and keystore password.
  • Keystore algorithm.
  • Truststore and truststore password.
  • Truststore algorithm.

Start by creating an instance of keymanager factory.

keystore = KeyStore.getInstance("jks");
keystoreIn = new FileInputStream(keystore);
keystore.load(keystoreIn, keyStorePassword);
kmf = KeyManagerFactory.getInstance(keyStoreAlgorithm);
kmf.init(keystore, keyStorePassword);

Similarly, we need a trustmanager factory.

truststore = KeyStore.getInstance("jks");
truststoreIn = new FileInputStream(truststore);
truststore.load(truststoreIn, trustStorePassword);
tmf = TrustManagerFactory.getInstance(trustStoreAlgorithm);
tmf.init(truststore);

We now have the inputs to create the SSL Context and the SSL Engine. It can be done as below.

sslContext = SSLContext.getInstance("TLSv1.2");
sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);
SSLEngine sslengine = sslContext.createSSLEngine();

Here we use the protocol TLSv1.2, because it is the latest available in Java. Once we have the SSLContext ready, we need to create an SSL engine and add some properties to it such as protocols ciphersuites and specify whether the engine is to be used in client mode or server mode.

For the transport layer, we have two channel pipeline factories, one for the server – ServerChannelPipelineFactory and one for client – ClientChannelPipelineFactory. In our plugin, we need to extend both for the transport section.

In the ServerChannelPipelineFactory, we need to specify that the SSL engine is used as the server with:

sslengine.setUseClientMode(false);

In ClientChannelPipelineFactory we need to set it to true. In both pipeline factories, add the protocols and ciphersuites to engine with:

sslengine.setEnabledProtocols(DEFAULT _PROTOCOLS);
sslengine.setEnabledCipherSuites(DEFAULT_CIPHERS);

You can decide which protocols or Ciphers to be used in your system for your needs. For example:

DEFAULT _PROTOCOLS = { "TLSv1", "TLSv1.1", "TLSv1.2" };
DEFAULT_CIPHERS = {"TLS_RSA_WITH_AES_128_CBC_SHA256", "TLS_RSA_WITH_AES_128_CBC_SHA"};

The getPipeline() needs to be overridden in our ChannelPiplelineFactories, which will allow us to add the sslhandler into the netty’s pipeline as shown below.

super.getPipeline().addFirst("ssl", new SslHandler(engine));

In the case of our ClientChannelPipelineFactorywe need to add the handshake to the pipeline along with the sslhandler, which performs the ssl handshake. For that, we need to have a class, CustomMessageChannelHandler, that extends the SimpleChannelHandler then overrides channelConnected(). In the channelConnected we need to do the ssl handshake and if the handshake is successful, send it to upstream channel.

SslHandler sslHandler = ctx.getPipeline().get(SslHandler.class);
final ChannelFuture handshakeFuture = sslHandler.handshake();
handshakeFuture.addListener(new ChannelFutureListener() {
               @Override
               public void operationComplete(ChannelFuture future) throws Exception {
                   if (future.isSuccess()) {
ctx.sendUpstream(e);
                   } else {                    
future.getChannel().close();                    
                   }
               }
        });

This CustomMessageChannelHandler needs to be added into the pipeline of ClientChannelPipelineFactory:

pipeline.addAfter("ssl", "handshake", new CustomMessageChannelHandler());

In your plugin class, both the custom NettyTransport and NettyHttpServerTransport classes need to be added to the respective modules so that it gets called as default channel.

TransportModule.setTransport(CustomNettyTransport.class, "ssl");
HttpServerModule.setHttpServerTransport(CustomNettyHttpServerTransport.class, "ssl");

The properties such as keystore, truststore, algorithms, ciphers, etc., can all be made configurable by adding these as properties in the elasticsearch.yml file and read them in the plugin class using elasticsearch settings. You may also provide configuration to enable/disable SSL, so that it modifies the pipeline only if it is enabled.

Package the plugin along with the plugin properties file and install into your elasticsearch server. We are now ready to have encryption enabled communication between the application and elasticsearch. When you next start elasticsearch, you will notice that the transport and http networks will be overridden by your custom plugin. The port used will be same as HTTP for HTTPS as well, because we are using the same channel for SSL.

It is recommended to do research on the algorithms and ciphers to be used. To generate the keystore and truststore you can check out the Oracle documentation.

Conclusion

This article went the Netty-way to implement the security plugin for your elasticsearch cluster. Now that we’ve secured the communication in our network, the next step is to authenticate elasticsearch users and later to authorize them. Look out for our cupcoming post on having authentication implemented in elasticsearch cluster.

Related Helpful Resources

Give It a Whirl!

It’s easy to spin up a standard hosted Elasticsearch cluster on any of our 47 Rackspace, Softlayer, Amazon or Microsoft Azure data centers, and easily select a ready-to-go SSL/TLS Security implementation. And you can now provision a replicated cluster.

Questions? Drop us a note, and we’ll get you a prompt response.

Not yet enjoying the benefits of a hosted ELK-stack enterprise search on Qbox? We invite you to create an account today and discover how easy it is to manage and scale your Elasticsearch environment in our cloud hosting service.