A common use case that comes up when we use any product is how can we get metrics from it? How can we monitor it? Elasticsearch, since its early release, has always provided a way to monitor it using the _cat/stats API. However, for Logstash there wasn’t a way to gather metrics and monitor it until recently. With the release of Logstash 5.0+, Logstash has introduced a set of APIs to monitor Logstash.  In this article we explore the monitoring APIs exposed by Logstash, which includes the Node Info API, the Plugins API, the Node Stats API, and the Hot Threads API. 

Keep reading

A question that we answer quite often is: What's the best way to monitor key performance metrics in Elasticsearch—such as response time? There are several open source projects for #Elasticsearch monitoring tools, and one very good commercial solution. In this article, we invite you to take three minutes our of your day to acquaint yourself with the best monitoring tools.

Keep reading

From Day One, Qbox has offered a number of monitoring plugins on our Elasticsearch cluster dashboard.

However, Marvel also has the same limitation of the other monitoring plugins: an engineer or tech must monitor the metrics manually to get event notifications. Qbox closes this gap—and establishes a lead-in to other upcoming features—with our new active cluster alerting feature.

We now have a system component that continuously checks clusters for potential problems, then creates an event and notifies users of issues. Continue reading to learn about the types of cluster alerts that we send.

Keep reading

During our recent trip to Boston for an Elasticsearch meetup, we heard a great talk by Xiao Yu on the challenges presented by large Elasticsearch clusters. If you've used a distributed data store, you probably can relate to the difficulty of managing several nodes. Xiao (@HypertextRanch), a Boston dev and blogger, knows all about those challenges -- and he created a new Elasticsearch plugin to deal with them.

Keep reading

Elasticsearch Marvel Released

Posted by Michael Lussier January 28, 2014

Elasticsearch has released Marvel, a cluster management and monitoring product built by members of the team itself. Marvel is an incredibly detailed system to monitor your cluster events providing deep insight into real-time metrics on your cluster, node, and indices. Marvel keeps you ten steps ahead allowing you to see where problems will arise. Giving you the heads up on every event and exactly what was affected. Even when you need to travel back in time, Marvel has you fully prepared, storing all historical metrics for you to view anytime. This allows for more time developing and less time dealing with cluster headaches.

Keep reading

Qbox devs just can't get enough Elasticsearch plugins. We especially seem to love web-based plugins that allow you to monitor/interact with your cluster.

Previously, we've offered plugins such as Paramedic, Elastic-HQ, and Bigdesk, to name a few. Today we expanded the list, adding a more recent discovery of ours, Kopf from lmenezes. If you're creating or editing a cluster, you'll see it as an option under "Monitor plugin".

Keep reading