REST Calls Made Easy – A New Elasticsearch Java Rest Client
Posted by Adam Vanderbush February 21, 2017How do you easily integrate Elasticsearch to your application? Elasticsearch gives us two ways, REST APIs and Native clients.
Which is the better solution? Like everything, there are pros and cons to both. For the REST APIs provided by Elasticsearch, you have to use third party libraries like JAX-RS to carry out the interaction. Although native clients are an easy option that come in many languages like Java, Python, Ruby, problems occur whenever there is a major version upgrade of Elasticsearch. You have to upgrade your native client, and many deem this as an unnecessary maintenance effort.
Luckily, things have gotten better with the release of Java REST Client. Apart from making REST calls, it also provides additional benefits such as:
- Load balancing across all available nodes.
- Failover in case of node failures and upon specific response codes.
- Persistent connections.
- Trace logging of requests and responses.
- Optional automatic discovery of cluster node.
- Failed connection penalization.
All these benefits and zero dependency on the version of Elasticsearch! This article explores the Java REST Client in detail.
Tutorial
For this post, we will be using hosted Elasticsearch on Qbox.io. You can sign up or launch your cluster here, or click “Get Started” in the header navigation. If you need help setting up, refer to “Provisioning a Qbox Elasticsearch Cluster.“
The Java REST client internally uses Apache HTTP Async Client to send HTTP requests. Because the Java Rest Client is hosted on Maven Central, to get started you have to add the following dependency in your pom.xml
file.
<dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>rest</artifactId> <version>5.0.0-rc1</version> </dependency>
To make any calls to Elasticsearch, we need to create a RestClient. This can be done using the RestClientBuilder class. Like most builder classes, it allows us to configure parameters while building the instance. Here, you can set values using the following setter methods: setDefaultHeaders
, setMaxRetryTimeoutMillis
, setFailureListener
, setRequestConfigCallback
, and setHttpClientConfigCallback
. The above setters are optional; the only required argument for RestClient.builder
is one or more HTTPHost instances.
Here is an example of creating a RestClient with only required arguments:
RestClient restClient = RestClient.builder( new HttpHost("localhost", 9200, "http"), new HttpHost("localhost", 9205, "http")).build();
If your Elasticsearch is secure, set the user credentials, too. If the Elasticsearch calls are going to be intercepted by some custom plugins you’ve written, don’t forget to set the appropriate headers. The “Header”
array can be instantiated and passed to the setter method setDefaultHeaders()
. Similarly, one can achieve basic authentication by providing an HttpClientConfigCallback
while building the RestClient through its builder. Below is an example that illustrates this:
Header[] headers = { new BasicHeader(HttpHeaders.CONTENT_TYPE, "application/json"), new BasicHeader("Role", "Read") }; final CredentialsProvider credentialsProvider = new BasicCredentialsProvider(); credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("admin", "admin")); restClient = RestClient.builder(new HttpHost("localhost", 9200)).setDefaultHeaders(headers) .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() { public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder arg0) { return arg0.setDefaultCredentialsProvider(credentialsProvider); } }) .build();
Elasticsearch supports both synchronous and asynchronous REST call requests via the performRequest
and performRequestAsync method variants of the RestClient respectively. These methods take HTTP verb and endpoint as arguments. There are other overloaded functions which accepts additional arguments like:
Map
to send query string parameters.HTTPEntity
object to send request body.ResponseListener
method that needs to notified on asynchronous request success or failures.HttpAsyncResponseConsumer
which controls how the response body gets streamed from a non-blocking HTTP connection on the client side.- An optional
Headers
array to pass on request headers.
Examples of the features of RestClient
Index a New Document
HttpEntity entity = new NStringEntity( "{\n" + " \"company\" : \"qbox\",\n" + " \"title\" : \"Elasticsearch rest client\"\n" + "}", ContentType.APPLICATION_JSON); Response indexResponse = restClient.performRequest( "PUT", "/blog/post/1", Collections.<String, String>emptyMap(), entity); System.out.println(EntityUtils.toString(indexResponse.getEntity()));
Output:
{"_index":"blog","_type":"post","_id":"1","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true}
Search the document using Query Params
Map<String, String> paramMap = new HashMap<String, String>(); paramMap.put("q", "company:qbox"); paramMap.put("pretty", "true"); Response response = restClient.performRequest("GET", "/blog/_search", paramMap); System.out.println(EntityUtils.toString(response.getEntity())); System.out.println("Host -" + response.getHost() ); System.out.println("RequestLine -"+ response.getRequestLine() );
Output:
{ "took" : 4, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 0.30685282, "hits" : [ { "_index" : "blog", "_type" : "post", "_id" : "1", "_score" : 0.30685282, "_source" : { "company" : "qbox", "title" : "Elasticsearch rest client" } } ] } } Host - http://localhost:9200 RequestLine - GET /blog/_search?q=company%3Aqbox&pretty=true HTTP/1.1
Apart from the general response output, we are also able to print details like hostname, request URL, et al. The Response object exposes details like these, which can be obtained by using its getter methods: getHeaders
, getStatusLine
, getEntity
, getRequestLine
, and getHost
.
Search the document using Query DSL
HttpEntity entity1 = new NStringEntity( "{\n" + " \"query\" : {\n" + " \"match\": { \"company\":\"qbox\"} \n" + "} \n"+ "}", ContentType.APPLICATION_JSON); Response response = restClient.performRequest("GET", "/blog/_search",Collections.singletonMap("pretty", "true"), entity1); System.out.println(EntityUtils.toString(response.getEntity()));
Output:
{ "took" : 3, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 0.30685282, "hits" : [ { "_index" : "blog", "_type" : "post", "_id" : "1", "_score" : 0.30685282, "_source" : { "company" : "qbox", "title" : "Elasticsearch rest client" } } ] } }
There could be several scenarios where you have to make Asynchronous REST calls. For an example, Say you have an aggregation operation that needs to be run on millions of documents. In this case, you don’t want this time-consuming operation to block your normal code execution flow. The async variant of RestClient i.e performRequestAsync
helps us in such cases. The following example shows you how to make async requests:
HttpEntity entity1 = new NStringEntity( "{\n" + " \"company\" : \"qbox\",\n" + " \"title\" : \"Elasticsearch rest client\"\n" + "}", ContentType.APPLICATION_JSON); HttpEntity entity2 = new NStringEntity( "{\n" + " \"company\" : \"supergaint\",\n" + " \"title\" : \"supergaint is awesome\"\n" + "}", ContentType.APPLICATION_JSON); HttpEntity entity3 = new NStringEntity( "{\n" + " \"company\" : \"linux foundation\",\n" + " \"title\" : \"join linux foundation\"\n" + "}", ContentType.APPLICATION_JSON); HttpEntity[] entityArray= {entity1, entity2,entity3}; int numRequests = 3; final CountDownLatch latch = new CountDownLatch(numRequests); for (int i = 0; i < numRequests; i++) { restClient.performRequestAsync( "PUT", "/blog/posts/" + i, Collections.<String, String>emptyMap(), entityArray[i], new ResponseListener() { @Override public void onSuccess(Response response) { System.out.println(response); latch.countDown(); } @Override public void onFailure(Exception exception) { System.out.println(exception.getMessage()); latch.countDown(); } } ); } //wait for completion of all requests latch.await();
Output:
Response{requestLine=PUT /blog/posts/1 HTTP/1.1, host=http://localhost:9200, response=HTTP/1.1 201 Created} Response{requestLine=PUT /blog/posts/2 HTTP/1.1, host=http://localhost:9200, response=HTTP/1.1 201 Created} Response{requestLine=PUT /blog/posts/0 HTTP/1.1, host=http://localhost:9200, response=HTTP/1.1 201 Created}
Conclusion
This is by far the easiest solution with the least maintenance effort given by Elasticsearch. In this article, we have stuck to the needs of a developer who wants to integrate Elasticsearch to his application. There is more to this REST client, features like sniffing, load balancing et al, which we will cover soon.
Related Helpful Resources
- Optimizing Elasticsearch: How Many Shards per Index?
- What is Elasticsearch, and How Can I Use It?
- Welcome to the ELK Stack: Elasticsearch, Logstash, and Kibana
- How to Integrate Slack with Elasticsearch, Logstash, and Kibana
- Microservices, Supergiant Architecture for Stability and Scale
Give It a Whirl!
It’s easy to spin up a standard hosted Elasticsearch cluster on any of our 47 Rackspace, Softlayer, Amazon or Microsoft Azure data centers. And you can now provision a replicated cluster.
Questions? Drop us a note, and we’ll get you a prompt response.
Not yet enjoying the benefits of a hosted ELK-stack enterprise search on Qbox? We invite you to create an account today and discover how easy it is to manage and scale your Elasticsearch environment in our cloud hosting service.