Your first reaction might be, “why not use the Timelion plugin or more recently Visual Builder with Kibana instead?”. We understand that Timelion is a good step towards turning Kibana into a legitimate Time Series Database (TSDB), however it still has a ways to go. It will be interesting to see how Timelion closes the gap in this department. 

The sheer options and flexibility to manipulate the data into gorgeous visualizations coupled with the open source community’s pre-made dashboards, are something that makes Grafana an excellent choice or alternative to Kibana’s offerings.

Install Grafana

We’re assuming you already have ELK setup on a Qbox cluster. At the time of this writing the latest stable build of Grafana is version 4.4.2. For this guide, we used Ubuntu 14.04 installation. 

First, add the following repo to your /etc/apt/sources.list file:

deb https://packagecloud.io/grafana/stable/debian/ jessie main

If you want to try betas or other release candidates add this repo instead:

deb https://packagecloud.io/grafana/testing/debian/ jessie main

Next, add the Package Cloud key to install the signed packages:

curl https://packagecloud.io/gpg.key | sudo apt-key add -

Update the Apt repositories and install the `grafana` package:

sudo apt-get update && sudo apt-get install grafana

Start the Service (systemd)

sudo systemctl daemon-reload && sudo systemctl start grafana-server && sudo systemctl status grafana-server

Enable the systemd service so Grafana starts at boot.

sudo systemctl enable grafana-server.service

Open Grafana

Open your browser-of-choice at http://<host>:3000 and use admin as both the username and password credentials.

Grafana Dashboard Overview


Connect to Elasticsearch

You will need to determine which elasticsearch indices you want to use to display in Grafana.

From the terminal, curl your Elasticsearch cluster to retrieve available indices. Use the following format: 

curl -XGET 'YOUR_CLUSTER_IP:PORT/_cat/indices?v&pretty'

Since our cluster is installed locally, we just ran:

curl -XGET 'localhost:9200/_cat/indices?v&pretty'
health status index                 uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   metricbeat-2017.08.06 EUiC49y4TYSRe5xud8L04w   5   1        150            0    282.7kb        282.7kb
yellow open   filebeat-2017.08.06   u7-6kYYHTMWjvo_5t_qqiw   5   1      21115            0      7.3mb          7.3mb
yellow open   .kibana               QofT1bBxQk6ATXzKMk8oeA   1   1          2            0     19.7kb         19.7kb

In this tutorial, I will be be using the two indices generated from metricbeat and filebeat.

Back in the Grafana dashboard, go to “Datasources > Add New,” name the source, and specify “Elasticsearch” as the data source type.

Grafana: connect to Elasticsearch


Enter the Elasticsearch cluster url, credentials to access the cluster (if needed), and  enter the name of the index you want to integrate from the curl command earlier. Click Save & Test to save the data source. Then, from the Grafana menu icon in the top left, select  Dashboards > New.

Create a Dashboard

From here, Grafana presents you with a nice selection of visualization types called “panels.”

Grafana: create a dashboard

Let’s create a Graph panel. A sample panel with dummy data will appear, but we want real data, so click on Panel Title > Edit where you will then be presented with numerous tabs to define various things. Go ahead and nuke the ‘Test data:random walk’ dummy source with the trashcan icon, and click a real data source from the Panel Data Source flip-down.

Grafana: choosing data source


I am using the Metricbeat datasource that we created earlier. Click Add Query. If you are a Kibana user, the query field to define Lucene queries should seem familiar to you. For this example, I am graphing the system load of the specified host using the ‘system.load’ field as provided by the official Metricbeat documentation.

I adjusted the metrics to ‘Max,’ and Group by time interval to ‘30s’. I then specified the time range to show the system load of the host for the past 15 minutes in the top right hand corner of the dashboard. It would be wise to familiarize yourself with the other tabs as well, adjusting the draw options under Display for a more robust data visualization.

Grafana System Load Visualization


Nice job! You have successfully integrated Grafana with Elasticsearch. Now, quickly get a snapshot of what Grafana is capable above, at least from a performance metrics visualization standpoint.

Conclusion

Snazzy right? For the next Grafana + Elasticsearch tutorial, we will explore in greater detail Metricbeat’s integration with Grafana, monitoring, and its other available modules.

Other Helpful Tutorials

Give It a Whirl!

It's easy to spin up a standard hosted Elasticsearch cluster on any of our 47 Rackspace, Softlayer, or Amazon data centers. And you can now provision your own AWS Credits on Qbox Private Hosted Elasticsearch

Questions? Drop us a note, and we'll get you a prompt response.

Not yet enjoying the benefits of a hosted ELK-stack enterprise search on Qbox? We invite you to create an account today and discover how easy it is to manage and scale your Elasticsearch environment in our cloud hosting service.