Qbox to join forces with Instaclustr. Read about it on our blog post here.

REST Calls Made Easy – A New Elasticsearch Java Rest Client

Posted by Adam Vanderbush February 21, 2017

How do you easily integrate Elasticsearch to your application? Elasticsearch gives us two ways, REST APIs and Native clients.

Which is the better solution? Like everything, there are pros and cons to both. For the REST APIs provided by Elasticsearch, you have to use third party libraries like JAX-RS to carry out the interaction. Although native clients are an easy option that come in many languages like Java, Python, Ruby, problems occur whenever there is a major version upgrade of Elasticsearch. You have to upgrade your native client, and many deem this as an unnecessary maintenance effort.

Luckily, things have gotten better with the release of Java REST Client. Apart from making REST calls, it also provides additional benefits such as:

  • Load balancing across all available nodes.
  • Failover in case of node failures and upon specific response codes.
  • Persistent connections.
  • Trace logging of requests and responses.
  • Optional automatic discovery of cluster node.
  • Failed connection penalization.

All these benefits and zero dependency on the version of Elasticsearch! This article explores the Java REST Client in detail.

Tutorial

For this post, we will be using hosted Elasticsearch on Qbox.io. You can sign up or launch your cluster here, or click “Get Started” in the header navigation. If you need help setting up, refer to “Provisioning a Qbox Elasticsearch Cluster.“

The Java REST client internally uses Apache HTTP Async Client to send HTTP requests. Because the Java Rest Client is hosted on Maven Central, to get started you have to add the following dependency in your pom.xml file.

<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>rest</artifactId>
    <version>5.0.0-rc1</version>
</dependency>

To make any calls to Elasticsearch, we need to create a RestClient. This can be done using the RestClientBuilder class. Like most builder classes, it allows us to configure parameters while building the instance. Here, you can set values using the following setter methods: setDefaultHeaders, setMaxRetryTimeoutMillis, setFailureListener, setRequestConfigCallback, and setHttpClientConfigCallback. The above setters are optional; the only required argument for RestClient.builder is one or more HTTPHost instances.

Here is an example of creating a RestClient with only required arguments:

RestClient restClient = RestClient.builder(
       new HttpHost("localhost", 9200, "http"),
       new HttpHost("localhost", 9205, "http")).build();

If your Elasticsearch is secure, set the user credentials, too. If the Elasticsearch calls are going to be intercepted by some custom plugins you’ve written, don’t forget to set the appropriate headers. The “Header” array can be instantiated and passed to the setter method setDefaultHeaders(). Similarly, one can achieve basic authentication by providing an HttpClientConfigCallback while building the RestClient through its builder. Below is an example that illustrates this:

Header[] headers = { new BasicHeader(HttpHeaders.CONTENT_TYPE, "application/json"),
                                                                               new BasicHeader("Role", "Read") };
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
                                               credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("admin", "admin"));
                                               restClient = RestClient.builder(new HttpHost("localhost", 9200)).setDefaultHeaders(headers)
.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
          public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder arg0) {
                                                                                                           
     return arg0.setDefaultCredentialsProvider(credentialsProvider);
               }
                })
                .build();

Elasticsearch supports both synchronous and asynchronous REST call requests via the performRequest and performRequestAsync method variants of the RestClient respectively. These methods take HTTP verb and endpoint as arguments. There are other overloaded functions which accepts additional arguments like:

  • Map to send query string parameters.
  • HTTPEntity object to send request body.
  • ResponseListener method that needs to notified on asynchronous request success or failures.
  • HttpAsyncResponseConsumer which controls how the response body gets streamed from a non-blocking HTTP connection on the client side.
  • An optional Headers array to pass on request headers.

Examples of the features of RestClient

Index a New Document

HttpEntity entity = new NStringEntity(
"{\n" +
"    \"company\" : \"qbox\",\n" +                                      
"    \"title\" : \"Elasticsearch rest client\"\n" +
"}", ContentType.APPLICATION_JSON);
Response indexResponse = restClient.performRequest(
"PUT",
"/blog/post/1",
Collections.<String, String>emptyMap(),
entity);
                              System.out.println(EntityUtils.toString(indexResponse.getEntity()));

Output:

{"_index":"blog","_type":"post","_id":"1","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true}

Search the document using Query Params

Map<String, String> paramMap = new HashMap<String, String>();
paramMap.put("q", "company:qbox");
paramMap.put("pretty", "true");
                               
Response response = restClient.performRequest("GET", "/blog/_search",
                                                           paramMap);
System.out.println(EntityUtils.toString(response.getEntity()));
System.out.println("Host -" + response.getHost() );
System.out.println("RequestLine -"+ response.getRequestLine() );

Output:

{
 "took" : 4,
 "timed_out" : false,
 "_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
 },
 "hits" : {
"total" : 1,
"max_score" : 0.30685282,
"hits" : [ {
     "_index" : "blog",
     "_type" : "post",
     "_id" : "1",
     "_score" : 0.30685282,
     "_source" : {
       "company" : "qbox",
       "title" : "Elasticsearch rest client"
     }
} ]
 }
}
Host - http://localhost:9200
RequestLine - GET /blog/_search?q=company%3Aqbox&pretty=true HTTP/1.1

Apart from the general response output, we are also able to print details like hostname, request URL, et al. The Response object exposes details like these, which can be obtained by using its getter methods: getHeaders, getStatusLine, getEntity, getRequestLine, and getHost.

Search the document using Query DSL

HttpEntity entity1 = new NStringEntity(
 "{\n" +
"    \"query\" : {\n" +
"    \"match\": { \"company\":\"qbox\"} \n" +
"} \n"+
"}", ContentType.APPLICATION_JSON);
                                                               
Response response = restClient.performRequest("GET", "/blog/_search",Collections.singletonMap("pretty", "true"),
                                                           entity1);
System.out.println(EntityUtils.toString(response.getEntity()));

Output:

{
 "took" : 3,
 "timed_out" : false,
 "_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
 },
 "hits" : {
"total" : 1,
"max_score" : 0.30685282,
"hits" : [ {
     "_index" : "blog",
     "_type" : "post",
     "_id" : "1",
     "_score" : 0.30685282,
     "_source" : {
       "company" : "qbox",
       "title" : "Elasticsearch rest client"
     }
} ]
 }
}

There could be several scenarios where you have to make Asynchronous REST calls. For an example, Say you have an aggregation operation that needs to be run on millions of documents. In this case, you don’t want this time-consuming operation to block your normal code execution flow. The async variant of RestClient i.e performRequestAsync helps us in such cases. The following example shows you how to make async requests:

HttpEntity entity1 = new NStringEntity(
"{\n" +
" \"company\" : \"qbox\",\n" +
" \"title\" : \"Elasticsearch rest client\"\n" +
"}", ContentType.APPLICATION_JSON);
HttpEntity entity2 = new NStringEntity(
"{\n" +
" \"company\" : \"supergaint\",\n" +
" \"title\" : \"supergaint is awesome\"\n" +
"}", ContentType.APPLICATION_JSON);
HttpEntity entity3 = new NStringEntity(
"{\n" +
" \"company\" : \"linux foundation\",\n" +
" \"title\" : \"join linux foundation\"\n" +
"}", ContentType.APPLICATION_JSON);
HttpEntity[] entityArray= {entity1, entity2,entity3};
int numRequests = 3;
final CountDownLatch latch = new CountDownLatch(numRequests);
for (int i = 0; i < numRequests; i++) {
restClient.performRequestAsync(
"PUT",
"/blog/posts/" + i,
Collections.<String, String>emptyMap(),
entityArray[i],
new ResponseListener() {
@Override
public void onSuccess(Response response) {
 System.out.println(response);
 latch.countDown();
}
@Override
public void onFailure(Exception exception) {
        System.out.println(exception.getMessage());
 latch.countDown();
}
}
);
}
//wait for completion of all requests
latch.await();

Output:

Response{requestLine=PUT /blog/posts/1 HTTP/1.1, host=http://localhost:9200, response=HTTP/1.1 201 Created}
Response{requestLine=PUT /blog/posts/2 HTTP/1.1, host=http://localhost:9200, response=HTTP/1.1 201 Created}
Response{requestLine=PUT /blog/posts/0 HTTP/1.1, host=http://localhost:9200, response=HTTP/1.1 201 Created}

Conclusion

This is by far the easiest solution with the least maintenance effort given by Elasticsearch. In this article, we have stuck to the needs of a developer who wants to integrate Elasticsearch to his application. There is more to this REST client, features like sniffing, load balancing et al, which we will cover soon.

Related Helpful Resources

Give It a Whirl!

It’s easy to spin up a standard hosted Elasticsearch cluster on any of our 47 Rackspace, Softlayer, Amazon or Microsoft Azure data centers. And you can now provision a replicated cluster.

Questions? Drop us a note, and we’ll get you a prompt response.

Not yet enjoying the benefits of a hosted ELK-stack enterprise search on Qbox? We invite you to create an account today and discover how easy it is to manage and scale your Elasticsearch environment in our cloud hosting service.

Categorized under:

Not yet enjoying the benefits of a hosted ELK-stack enterprise search on Qbox? Discover how easy it is to manage and scale your Elasticsearch environment.

Get Started 5 minutes to get started