We cannot over-emphasize that the ELK stack is a great solution to ship, search, and analyze logs, system metrics, statistics, and other types of insight-driven data. You can utilize various components of the ELK stack such as Kibana to monitor what is happening in your cluster/s, host and applications getting instant insights guiding your business decisions. 

However, what options do we have for monitoring Elasticsearch itself? To make Elasticsearch serve request fast and ensure the health of the cluster, we need a good monitoring solution that helps identify issues as they arise. Fortunately, there are a lot of free monitoring tools available for the Elasticsearch, including Elasticsearch Kopf , Big Desk, or Whatson

In this article, we'll review one of the best web-based monitoring tools for Elasticsearch -- ElasticHQ. This plugin has been chosen as the built-in monitoring solution by Qbox for its hosted Elasticsearch 6.2.1 clusters.

Keep reading

Logstash ships with many input, codec, filter, and output plugins that can be used to retrieve, transform, filter, and send logs and events from various applications, servers, and network channels. 

In the previous tutorials, we discussed how to use Logstash to ship Redis logsindex emails using Logstash IMAP input plugin, and many other use cases. 

In this article, we continue our journey into the rich world of Logstash input plugins focusing on the Beats family (e.g., Filebeat and Metricbeat), various file and system input plugins, network, email, and chat protocols, cloud platforms, web applications, and message brokers/platforms. Logstash currently supports over 50 input plugin -- and more are coming -- so covering all of them in one article is not possible. Therefore, we decided to overview some of the most popular input plugin categories to give you a general picture of what you can do with Logstash. 

Keep reading

Not yet enjoying the benefits of a hosted ELK-stack enterprise search on Qbox? Discover how easy it is to manage and scale your Elasticsearch environment.

Get Started 5 minutes to get started

So you have moved all your applications to Docker and have begun enjoying all the fruits of lightweight and fast-to-deploy containers. 

That's great, but once you have multiple containers spread across multiple nodes, you'll need to find a way to track their health, storage, CPU, and memory usage, network load, etc. 

To track these metrics, you need an efficient monitoring solution and some backend store to keep your container data for subsequent analysis and processing. Managing thousands of Docker containers in production made our team here at Qbox quickly realize that Docker container monitoring is a valuable addition to our cluster management process. 

In a previous article, we discussed how to use Metricbeat to ship metrics from Kubernetes. Now, it's time to share our experience of using Metricbeat to monitor bare Docker containers and shipping container data to Elasticsearch and Kibana. This knowledge may be useful for developers and administrators who manage Docker containers without orchestration. Let's get started!

Keep reading

Kubernetes is a popular container orchestration and container management platform for automating deployment, scheduling, and update of your containerized workloads in distributed compute environments. It goes without saying that managing multiple nodes and applications in Kubernetes requires an efficient monitoring system. You need to have a real-time picture of events happening in your cluster to get actionable insights for optimization and improving performance. 

Kubernetes ships with some default monitoring and metrics solutions like Heapster and Kubernetes Metrics Server. However, in order to apply analytics, do data discovery, and visualize metrics data flowing from your cluster, you'll be better off using solutions designed specifically for such type of tasks. One popular option for log and metrics monitoring and analysis is the ELK stack (Elasticsearch, Logstash, Kibana) used in pair with Elastic Beats log shippers. 

In this article, we introduce you to monitoring Kubernetes with ELK and Elastic Beats. In particular, we'll show how to send Kubernetes metrics to Elasticsearch indexes using Metricbeat and access them in your Kibana dashboard for subsequent processing. Let's get started!

Keep reading

Our Commitment to GDPR

Posted by Mark Brandon May 24, 2018

Qbox is fully committed to achieving compliance with the General Data Protection Regulation (GDPR) by the May 25 deadline.

Keep reading

Bulk indexing in Elasticsearch is an important topic to understand because you might occasionally need to write your own code to bulk index custom data. In addition, experience with bulk indexing is important when you need to understand performance issues with an Elasticsearch cluster.

Keep reading