Qbox to join forces with Instaclustr. Read about it on our blog post here.

We set up for this post, Part 3 in our Kibana tutorial blog series, by working through the basics of the setup and usage our Kibana analytics dashboard in Using Kibana and Elasticsearch to Examine Twitter Trends(Part 1) and in Tutorial: Simple Analysis using Kibana (Part 2).

<p”>Here, we’ll take a look at some of the advanced analytics features in Kibana, including </p”>

  • Maps analytics
  • <li”>Hits analytics </li”>
  • <li”>List analytics</li”>
  • <li”></li”>

First, let’s recap the previous two articles:

  • In the first article in this series, we set up an Elasticsearch cluster on qbox.io and sent a Twitter data river (of hashtags #Marvel) to the cluster. We also got a closer look at the data structures.
  • The second article covers access to the Kibana dashboard from the cluster and along with how to access the Twitter index (“comics”) that we created through Kibana dashboard.
  • The second article also covers the basic analytic techniques for Twitter hashtags, using histogram, pie charts, and table tools from our Kibana dashboard.

Now that we understand a few basic Kibana analytics techniques and know something about a simple Twitter data structure, let’s move on to those advanced analytics features in Kibana, as we promised.


1. Configuring the Kibana Dashboard

In the first article of this series, we saw how easy it is to establish an Elasticsearch cluster on qbox.io and from there to access the Kibana dashboard. As shown in Section 1 that article, we can create a new blank Kibana dashboard and name it by clicking the Settings icon in the top right of the dashboard as shown in Figure 1A below.

Figure 1A


We will give it the name “Advanced Kibana Analytics” as in Figure 1B below.

Figure 1B


Next, select the Index tab, enter a name for the index of the data river (“comics”), and check the Preload Fields box.

An important note about Preload Fields: Since there will be numerous fields in our index and they will appear often in our Kibana dashboard, we want to avoid wasting time by referring to the JSON data format of the index data each time, along with checking the field name and then typing it in required places. An easier method is to preload the fields with an autocomplete function. When we type the first letter of the field in a box, every field starting with that particular alphabet is shown in a context list from which a selection can be made. You can enable this feature by checking the Preload Fields box.

Figure 1C


Along the top, you will also find the Timepicker tab. Click that tab, and then locate the Time Field. Enter the name of time field that contains the data. In Figure 1D below, we will use the name “created_at” and then click the Save button to commit our changes and return to the Kibana dashboard.

Figure 1D


2. Map Analytics

The first Kibana analytics feature that we’ll cover here is the Map analytics tool, which displays a geographical map that contains regions with various degrees of shading. The intensity of the shading will correspond to the number of hits in a specific region. That is, a region having relatively more hits will have darker shading; those with relatively fewer hits will have lighter shading. Kibana currently offers three types of maps: a U.S. map for all 50 states; a map of Europe; and a world map. Let’s work through an example of creating a U.S. map.

U.S. Map Analytics

We begin by clicking the Add a Row tab in the bottom right of the dashboard, typing the name “Map Analytics,” and then clicking Create Row. See Figure 2A below.

Figure 2A


Now, when we return to the dashboard we find an empty row, as shown in Figure 2B.

Figure 2B


At this point, you can click the Add panel to the empty row button, which we highlight in the figure above. A configuration panel will appear, similar to that shown in the following figure.

Figure 2C


From the Select Panel Type dropdown, choose the “map” item in the listing. The panel will reveal many more items, as seen in Figure 2D.

Figure 2D


In the figure above you can see several elements that you can configure:

  • Title — Type in the title for your map based analytics, such as “U.S. map.”
  • Span — This is number of units of width that this map will occupy in the row.
  • Field — This is the most important element. Type in the name of the field in the source data that contains the location data that you want to map, such as gps cordinates, postal codes, country names, or place names. For our example here, the location data is stored in the location field, which falls within the user category. So we type user.location in the Field box.
  • Map — Click the dropdown and choose from three items: U.S. country map, Europe map, or the World map. For this example, we choose the U.S. map.

Click the Save button to commit your changes and return to the Kibana dashboard, which should appear as shown in Figure 2E.

Figure 2E


In this map we see differrent shades of the same color. The darker shading corresponds to more hits in that region. Also shown in the figure, hovering the mouse over a specific region such as Texas will display the number hits for that region.

We can add the Europe and World maps by clicking the Add Panel button (the green button near the top of the row). Refer to the figure below.

Figure 2F


After creating the Europe and World maps, your Map Analytics row would look like the one above. From the figure above, we can deduce that the most interests in the hashtags “marvel” and “comics” originate in the U.S., followed by Canada and then India.

Map Analytics Using Multiple Queries

With Kibana analytics tools, we can analyze the hits from more than one query. We saw how to do a single query in the Map Analytics Visualisation section, so let’s go further using the Kibana dashboard. Our current dashboard looks like the figure below.

Figure 2G


Click on the + on the far right side of the query panel to see another query box appear in the panel. Let’s add 3 query boxes in the panel. You will see that that Kibana automatically colors the query panels green, orange, and blue. Next, type in three differrent queries: “Godzilla,” ”Wolverine,” and “Robocop.” You can also remove the query search bar by clicking the small close icon (which is visible when you hover inside the query search bar). The dashboard will appear as shown in Figure 2H below.

Figure 2H


Click the Search button to see the transformation occur in “Map Analytics Row” as we show in the figure below.

Figure 2I


We see that there is no drastic differrence shown in the maps because multiple-query analytics is done using OR logic. If there are are no additional regions, then the combined shading will be shown in each of the regions.

3. Hits Analytics

The hits analytics feature is also very useful because it provides the total hits for one or more queries. You can also see a comparison of the hits for multiple queries. The hits analytics feature in Kibana offers the capability to visualize the data in four ways: bar graph, pie chart, list view, and the totals view.

First, let’s create a row by clicking the Add a row button on the bottom right of the Map Analytics row. To the right, we type the name of the new row in the Hits Analytics field, then click Create Row as shown in the Figure 3A below. Click the Save button to commit your changes.

Figure 3A


Bar Graphs Visualization

Let’s move on to the bar graphs visualization of hits analytics. After we complete the step above, the dashboard will appear as shown in Figure 3B.

Figure 3B


In this new row, let’s create a new panel for our bar graphs. Similar to what we saw in the steps above in the map analytics section, we click on the Add Panel tab (see the highlight in the figure above). Remember this step because we’ll have multiple panels in the same row.

After we click the Add Panel button, a new window will appear containing a Select Panel Type dropdown. Again, as we’ve done previously, we click the dropdown and choose Hits to display the window as shown in the next figure.

Figure 3C


These are the explanations for the settings in the figure:

  • Title — We use a simple title of “Bar Panel.”
  • Span — This is the same as we explain above. Here we choose a set the value to 3 because we want to divide the 12 sections equally so that everything fits into the same row.
  • Style — This is the style for the Hits analytics. We leave the default value of Bar.

Click the Save button and our dashboard will look like the figure below.

Figure 3D


Notice in Figure 3D above that the total number of hits is given as a single bar graph for an empty query. To get an even better grasp, let’s do another query. For example, let’s query for the word “valentines” in the Kibana query dashboard and press the Enter key. Now we get different results: a total of 517 hits come back, as you can see in the bar graph data stats in Figure 3E.

Figure 3E


Optionally, you can clear the query panel and hit search again to see the previous results.

Pie Graphs Visualization

Pie graph analytics will present the same data in a pie-chart. The setup is nearly the same as the bar chart, except for the Title and Style field. In our example that we depict in Figure 3F, we set the Title to “Pie Panel” and choose “Pie” from the Style dropdown.

Figure 3F


Click the Save button to commit your changes and return to the dashboard, which should now look like Figure 3G.

Figure 3G


Compare the bar chart and the pie chart to see that the information is the same; only the presentation is different.

List Analytics

In the Add Panel step, the setup is nearly the same as the for the pie and bar charts. The difference, again, is with the Title and Style field. In the example that we show in Figure 3H, we set the Title to “Lists Panel” and choose “List” from the Style dropdown.“>“>“>“>“>

Figure 3H


Click the Save button to commit your changes and return to the dashboard, which should look like Figure 3I below.

Figure 3I


Here we can see the same information that is found in the bar and pie charts, although it is simply given in tabular form.

Total Analytics

As we’ve done previously, let’s add another panel to the Hits Analytics row. Let’s set Title to “Total Panel” and Style to “Total,” as shown in Figure 3J.

Figure 3J


Click the Save button to display the Total Panel next to the Lists panel, as is shown in Figure 3K below.

Figure 3K


As you can see in Figure 3K above, the Total Panel displays the total hits for all the queries and is nearly identical to the Lists Panel. We’ll learn more about some slight differrences in the next section.

Hits Analytics – Multiple Queries

Now we can look into how multiple queries can be given in Hits Analytics. The setup for multiple queries is same as that given in the section Maps analytics using multiple queries above. In the example here, we’ll use the same queries (Godzilla, Wolverine, Robocop), and after we click the Search button,we’ll get the following stats in our Kibana dashboard.

Figure 3L


Here you can see that the hit count stats of differrent queries have different colors. The first three panels — the Bar Panel, Pie Panel, and List Panel –– show the hit counts of the three queries. The Total Panel shows the total hit counts of the three queries.


This concludes this post in which we addressed three of the advanced features in Kabana (maps analytics; hits analytics; and list analystics). Please note that the kind of analytics will of course vary according to the context and kind of the data that you’re analyzing.

Check back soon for Part 4 in this Advanced Kibana Analytics series to learn about other advanced analytics features in Kibana, including histogram analytics, trends analytics, and stats analytics.