We come now to the 4th article in our Kibana tutorial blog series. We covered three advanced analytics features (maps analytics, hits analytics, and list anaytics) in Advanced Kibana Analytics (Part 3 of Series). In this article we’ll look look at three more, including:
- Histogram analytics
- Trends analytics
- Stats analytics
4. Histogram Analytics
In part 2 of this tutorial series, we have saw how to do histogram analytics. Now, let’s take that to the next level using multiple queries.
After the initial configuration and setup of the Kibana dashboard, we need to create a new row named “Advanced Histogram.” As we’ve also shown above, let’s create a row and name it so that we get the result shown in the figure below.
Click the Create Row button and then the Save button. An empty row will appear. Now, as we have seen in the previous Kibana tutorial article, we need to set up a panel to display the histogram. Simply click the Add Panel button on the left hand side of the empty row and set the configuration to be like the one given below in Figure 4B.
Here’s an explanation of the values that we input to the important fields:
- Select Panel Type — To indicate which type of analytics, we set this to “Histogram.”
- Title — This field specifies a name for the panel, which we set as “Histogram Panel.”
- Time Field — This is the time field from our data set. In our data format, we had set the time field name to “created_at.” (Refer to part 1 of this series.)
- Span — We set the span value to 12 so that our histogram will occupy the entire row.
Click the Save button to display a histogram like the one in the figure below.
You might agree that the histogram above looks somewhat clumsy, and you also may recall from part 2 of this series that we have this problem because the default interval period for the histogram is quite long (one year). We can easily change this by clicking the View option and also changing the Interval value to a more suitable one. In the figure, you can see that we set the Interval to 3h (3 hours) in the dropdown.
After making these changes, we get a histogram as shown in Figure 4D.
Let’s go on a bit further with the histogram and type a simple query into the panel. We want to perform a query that covers a trend for the last few days. We could extend one of the examples from above, but that would limit the scope of our study because we’re likely to find little fluctuation for the histogram.
Take time now to review the pie chart setup involving the hashtag field of our tweets in our Kibana Tutorial Blog Part 2. Do the same simple analysis once more, but don’t bother saving the pie chart to your dashboard (because we will add in a few trending keywords).
At the time of writing this article, the pie chart analytics against our Twitter data gave me the “Avengers” and “Spiderman” as the top two trending words after “marvel” and “comics.” We will therefore use one of them to see its tweet histogram. Querying “Avengers” on my query panel in Kibana dashboard gives me the histogram shown in Figure 4E.
You’ll notice above that the variation in number of tweets mentioning “avengers” has suddenly jumped by February 7.
4a. Histogram Analytics Using Multiple Queries
In the previous section, we saw the behavior of Kibana histogram analytics using single queries. Now we are going to observe the variation in behavior and appearance when we submit multiple queries.
Let’s type in the keyword “spiderman” and press the Enter key. In the figure below, you can seen that the the number of tweets peak at about February 10.
On the right end of the query panel, press the + button to split the query search bar into two parts. Type “avengers” in the first query search box and type “spiderman” in the second one, then click the Search button. We get a histogram like the one shown in Figure 4G.
As you can see in the figure above, the resulting histogram is now a combination. We indicate regions 1 and 2 with red letters. Region 1 is the time at which there was sudden increase in the keyword “avengers,” and region 2 is when there was sudden increase in the keyword “superman.” The individual histograms have coloring for representation: green denotes the “avengers” histogram and orange denotes the “spiderman” historgram.
Look closely to see the “Spiderman” peak, which indicates much more interest in the Twitter community in comparison with “Avengers.” To see this, look at the “avengers” peak on February 7, which is only half that of the peak for “Spiderman” on February 10.
5. Trends Analytics
Trends is another useful Kibana tool. In this section, we’ll see how this feature calculates the rise and fall in the relative value of a particular query for a specific period.
Suppose that we want to know how the “avengers” is trending in the Twitter data in last one hour. Let’s have a look at how it can be done in Kibana. First, create a row having the title “Trends Row,” as we show in Figure 5A. Then click Create Row and then save it.
Next, we’ll add a new panel using the green button towards the left end of the new row. The result should look like the figure below.
In addition to the basic fields, we have two new fields:
- Time Ago — Choose the length of time that you want to go back into the past. For this example, we leave the default value of 1d (1 day).
- List Format — This asks for the type of listing of the trends. The trends are generally listed in stock-ticker style. Let it be the default value, i.e., vertical, which displays the values vertically as rows in a table.
After setting the fields, click Save to display the Trends panel on your Kibana dashboard as we depict in Figure 5C below.
In the above screenshot you can see an error message (indicated as 1 in red box) on the Trends panel saying “Oops!A time filter must exist for this panel to function.”
This error is the result of one of these causes:
- If, while configuring the dashboard, the Time field has not been set (in the Timepicker tab) to indicate our index (for our example here, the time field is “created_at”). We have set this earlier in the initial Kibana configuration (step 1D of our tutorial).
- Another reason for the error message is that the time filter option isn’t set. It’s important to specify the interval over which you want the analytics to be done. In the top right of the panel, click the Time Filter dropdown, to view differrent intervals of time. Let us select 1 hour as our interval (given as “Last 1h”).
After selecting the interval value, you’ll see results similar to Figure 5D.
In the figure above, we can see a decrease in interest of all the queries. We can see a drop of 15.87% during the 15:28pm to 16:28pm interval on the last day. Hover over the percentage value to display a small window (the #1 red box) showing the then/now values. This gives the previous and present values.
If you were to search for “Avengers” in the query box, you’d get the results shown in Figure 5E below.
Let’s describe the three highlights in the figure above:
- Filtering — The field shown in red box 1 is the filter that is applied.
- Then and Now Values — The values shown in red box 2 are the exact values of the start time and the current time.
- Percentage Value — Red box 3 contains the percentage increase or decrease for the current query.
5a. Trends Analytics with Multiple Queries
So far in this section, we’ve only done a single query. Now we’re going to apply multiple queries. Click on the + sign in the query search box to display two query search boxes.
In the second box, type “spiderman” and then click the Search button to see results in the query panel as shown in Figure 5F below.
Now we can clearly see the variation that results from multiple queries over a specific period. You can hover over each percentage value to see the then/now values.
When you’re done, don’t forget to save the Kibana dashboard. Simply click on the save icons (red boxes 2 and 3 above). It’s important to do this regularly because a refresh may wipe out all of your work.
6. Stats Analytics
Another great Kibana feature is stats analytics, which you can use to get various statistics for any numerical field. In our example, we have a couple of numerical fields. First, we’ll go after the retweet counts, which are given by retweet_count.
Before going into the creation of rows and panel, we need to do two things:
- Remove the filter conditions that we applied earlier to get the trends analytics. Simply click the close button (x) in the filtering box (redbox 1 in the Figure 6A).
- Remove all of the queries. Simply click the close icon in the search box (redbox 2 below). For the other, simply delete the query from the search box (redbox 3).
After removing the filters and the query, we create a new row “Stats Row” as shown below.
Next, create a new panel with the settings given in Figure 6C.
Take note of the Fields box (redbox 5). Type in the names of only those fields containing numerical values. For our example here, we type “retweets_count”.
Next, we want to choose an option from the Featured Stat dropdown (red box 4). Here the default value is count, which gives us the total count of any retweets. We can do many other operations, including min and max values, variance and the standard deviation.
Now click Save to see what appears in the Stats panel. The stats panel shows something like step 6D below.
In red box 1 above, we can see the total number of retweets in our data.
6a. Stats Analytics on Multiple Queries“>
As we done with other features, let’s go a bit further to include a second query: “avengers” and “spiderman” again. Searching on these queries gives the result below.
We can see the query name and the correspond retweet counts for each query. Red box 2 indicates the total of both counts.
We trust that this post and Part 3 of this series help you gain a better understanding of the advanced analytics features available in Kibana. In the next article in this series, we’ll cover some of the other tools.