Hosted Elasticsearch Service

The Authoritative Guide to Elasticsearch Search Tuning (Part 2)

This post is part 2 of a 3-part series about tuning Elasticsearch Search Tuning. Part 1 can be found here. The aim of this tutorial is to further discuss Search Tuning techniques, strategies and recommendations specific to Elasticsearch 5.0 or onwards.

Kibana, Logstash, Beats, Elasticsearch - are all version 5.0 now. It is the fastest, safest, most resilient, easiest to use version of Elasticsearch ever, and it comes with a boatload of enhancements and new features.

Read More

Data Retention Techniques in Logstash via Elasticsearch-Curator

Although elasticsearch can scale indefinitely, you should store required data only. This will speed up the search operation, as well as response time to retrieve the data, and even reduce resource utilization substantially.

Elasticsearch uses an “Inverted Index” to retrieve data that you are searching for. Although this algorithm is one of the best when it comes to text searching, keeping only the data that you need in the index is the best approach.

In this tutorial, we discuss data retention techniques that you can use in elasticsearch. This will obviously depend on the kind of data and your application, because some might need longer retention policies compared to others. 

Imagine an application that deals with finance and money transactions. Such applications will need all of the records forever. But, do these records need to always exist in elasticsearch? Does all of this data need to be quickly searchable?

Logstash provides methods where you can segregate different events, and then store it in standard file storage rather than elasticsearch for long-term storage.

Read More