We cannot over-emphasize that the ELK stack is a great solution to ship, search, and analyze logs, system metrics, statistics, and other types of insight-driven data. You can utilize various components of the ELK stack such as Kibana to monitor what is happening in your cluster/s, host and applications getting instant insights guiding your business decisions. 

However, what options do we have for monitoring Elasticsearch itself? To make Elasticsearch serve request fast and ensure the health of the cluster, we need a good monitoring solution that helps identify issues as they arise. Fortunately, there are a lot of free monitoring tools available for the Elasticsearch, including Elasticsearch Kopf , Big Desk, or Whatson

In this article, we'll review one of the best web-based monitoring tools for Elasticsearch -- ElasticHQ. This plugin has been chosen as the built-in monitoring solution by Qbox for its hosted Elasticsearch 6.2.1 clusters.

ElasticsearchHQ Core Features

ElasticHQ is a real-time monitoring and management tool that instantly informs developers and cluster administrators about what is happening in individual nodes, indices, and helps live-stream health information during heavy tasks. Along with diverse diagnostic tools to monitor various aspects of your cluster/s, ElasticHQ ships with a number of cluster management features like index management (creating, deleting and refreshing indices), index querying and search, index copying, and managing mappings. The plugin has support for all major Elasticsearch versions starting from Elasticsearch 2.x. All this goodness is packaged in the easy-to-use web-based UI making cluster monitoring and management simpler even for people with a modest technical background in Elasticsearch.

ElasticHQ is provided as a default monitoring solution on Qbox-hosted Elasticsearch 6.2.1 clusters. Under the hood, Qbox engine installs the plugin and configures it to work with your Elasticsearch cluster/s. After your Qbox-hosted cluster is up and running, you can easily access the ElasticHQ endpoint using your automatically generated credentials.

ElasticHQ in Qbox


If you are self-hosting Elasticsearch, you can easily install ElasticHQ from source or as a Docker container following these instructions.

After logging in into the ElasticHQ dashboard, the first thing you'll see is an overview of your Elasticsearch cluster:

ElasticHQ Cluster Overview


In the image above, you can see that we have only one node displayed in the left top part of the screen. However, ElasticHQ allows monitoring multiple nodes and clusters. From this view, you can also easily access your node/s statistics (left red arrow) and available indices (right red arrow). Below is the extended description of our node's state and statistics:

ElasticHQ Node statistics


Here you can see the node's summary including such useful statistics as the JVM heap usage, Virtual Memory (VM) usage, RAM and CPU usage.

In ElasticHQ, you can also access this data in the visualized form under the Metrics tab.

ElasticHQ Node Metrics


ElasticHQ also provides a detailed overview of your indices. Clicking on the individual index will open its detailed statistics like in the example below:

ElasticHQ Index stats

Along with the index stats, ElasticHQ provides a fine-grained control over your indices. In particular, it exposes such Elasticsearch index operations as index creation, flushing, refresh, index merge, clearing cache etc. These features can be accessed under the indexes view:

ElasticHQ Index operations

Or, alternatively, you can access these operations under the Administration tab of the individual index:

ElasticHQ Index operations


Although ElasticHQ is good in aggregating all important cluster metrics, sometimes you might need to directly access Elasticsearch REST API. For your convenience, ElasticHQ provides an easy to use REST API interface for accessing all basic Elasticsearch endpoints including _cat, cluster health, and custom HQ endpoints.


This feature is a valuable alternative to using curl and other terminal-based HTTP and REST API clients.

Conclusion

As you've seen, ElasticHQ is a powerful monitoring solution for your Elasticsearch cluster/s. The plugin is available by default in our Qbox-hosted 6.2.1 clusters but can be easily installed if you are hosting Elasticsearch on-premises or elsewhere. In addition to valuable cluster and node resource usage summaries and indices statistics, the tool offers a number of index and cluster management options, as well as the full access to Elasticsearch API from your browser. The plugin has multi-cluster support so you can have a full view of your multi-cloud Elasticsearch deployments under one web-based interface avoiding the overheads of monitoring each Elasticsearch deployment individually. Stay tuned to our new tutorials to learn more options for monitoring and managing your Elasticsearch clusters.