How to Use Elasticsearch with Kibana’s Time Series Visual Builder

It’s finally here. A legitimate contender to Grafana in the time series DB visualization space. Time series analysis with Kibana has been at least a few steps behind Grafana, even with Timelion. Time Series Visual Builder, however, levels the playing field with its own set of visualization customizations not seen in Timelion. In this tutorial, we will install and deploy Kibana and Metricbeat on a QBox cluster and play with the latest Visual Builder feature!

This tutorial assumes you have already spun up a QBox cluster. If not, sign up, and spin one up!

Install Kibana

$ sudo apt-get -y install kibana

Configure Kibana and Start Service

Ensure that the following parameters are uncommented and reflect the correct values in the Kibana configuration YML file at /etc/kibana/kibana.yml.

server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://localhost:9200"

Start Kibana & Allow Startup at Boot

$ sudo systemctl service kibana start && sudo systemctl enable kibana.service

Install Metricbeat

Let’s use Metricbeat to produce some time series data to play with. Metricbeat is a service that collects system metrics from your servers.

$ sudo apt-get -y install metricbeat

Before using Metricbeat service, configure the shipper in the metricbeat.yml file usually located under the /etc/metricbeat/ folder on Linux distributions. In the configuration file, at minimum specify Kibana’s and Elasticsearch’s hosts you want to send your data to and specify some modules from which you want Metricbeat to collect data. See Metricbeat documentation for more details about configuration.

Start the Metricbeat service:

sudo systemctl service metricbeat start && sudo systemctl enable metricbeat.service

Load Data in Kibana

Open your browser and access the Kibana dashboard at http://<QBoxClusterURL>:5601. In the left UI pane, go to Management > Index Patterns. Create an index pattern that reads metricbeat-*. Select the @timestamp filter in the Time Filter drop down menu.

Playing with the Visual Builder

From the left pane in Kibana, go to Visualize > Create A Visualization > Visual Builder (under Time Series). If you’re a Grafana user, you might be thrown off by the menu layout just a bit. Take note of the top menu of visualization types (Time Series, Metric, Top N, Gauge, Markdown), and familiarize yourself with the settings,and parameters within each menu.

Below is an example of a visualization I created for the max percentage of CPU time spent on low-priority processes (system.cpu.nice.pct). To produce the visualization, I used a max aggregation for the field system.cpu.nice.pct and specified a period of 1 hour to display. Metricbeat offers a lot of system metrics and parameters, so you can now refer and play around with then using examples from the official Metricbeat documentation.

Conclusion

KIbana’s new Visual Builder is a formidable visualization and graphing tool that comes right out-the-box, and the Grafana community has likely taken notice. That being said, however, it has taken quite some time for Kibana to release such a much overdue, critical feature as far as time series visualizations are concerned.

The open source nature of Grafana and the ever increasing development and support from community contributors, continues to make it an attractive, go-to tool. If Kibana were to launch a public repository for community-made dashboards, it could be another step closer to close the gap between the two, however. For now though, there is nothing keeping us from using both Kibana and Grafana for a variety of applications and purposes.

Related Helpful Resources

• The Authoritative Guide to Elasticsearch Performance Tuning (Part 1)
• How to Maximize Elasticsearch Indexing Performance (Part 1)
• Find Similar Documents in Elasticsearch with “More Like This Query”
• Elasticsearch Aggregations: Weekday and Hourly Analysis
• How to Index Attachments and Files to Elasticsearch

Give It a Whirl!

It’s easy to spin up a standard hosted Elasticsearch cluster on any of our 47 Rackspace, Softlayer, Amazon, or Microsoft Azure data centers. And you can now provision your own AWS Credits on Qbox Private Hosted Elasticsearch.

Questions? Drop us a note, and we’ll get you a prompt response.

Not yet enjoying the benefits of a hosted ELK-stack enterprise search on Qbox? We invite you to create an account today and discover how easy it is to manage and scale your Elasticsearch environment in our cloud hosting service.