Metrics Aggregations in Elasticsearch


preview imageProgramming
by Anurag Srivastava,Aug 18, 2018, 6:02:20 PM | 4 minutes |

In my previous blog, I have explained about basic aggregations. Now, let us pick the metrics aggregations and see how we can create these types of aggregations. Metrics aggregations are those aggregation where we apply different types of metrics on fields of Elasticsearch documents like min, max, avg, top, and stats etc.

Max:

I am going to explain you the max metrics using which we can get the max value for the given field. In the same way, we can use min,avg, and top etc. See below example

GET bqstack/_search?size=0
{
  "aggs": {
    "blog_metrics" : {
      "max" : {
        "field" : "views"
      }
    }
  }
}

In the above expression, I am trying to fetch the max number of views from all documents. After running this expression you may get the following result:

{
  "took": 9,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 54,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "blog_metrics": {
      "value": 6309
    }
  }
}

In the result, you can see that the value for blog_metrics is showing as 6309. In the query, we can pass as min or avg to get minimum or the average number of views.

Stats:

Stats provides us the consolidated stats metrics for a given field. Take another example where I am going to use stats instead of max:

GET bqstack/_search?size=0
{
  "aggs": {
    "blog_metrics" : {
      "stats" : {
        "field" : "views"
      }
    }
  }
}

In the above expression, I have replaced "max" keyword with "stats" rest all expression will remain the same. Now see the result of above expression.

{
  "took": 22,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 54,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "blog_metrics": {
      "count": 54,
      "min": 11,
      "max": 6309,
      "avg": 142.85185185185185,
      "sum": 7714
    }
  }
}

When I applied the "stats" keywords the aggregations result displayed all key stats like count, min, max, avg, and the sum of the given field value. Stats can be used when we want to see the data trend and want to know the min value, max value, sum, count and the average value of a field.

Extended Stats:
We can use extended_stats to get extended stats like sum_of_squares, variance, std deviation etc. See below example:

GET bqstack/_search?size=0
{
  "aggs": {
    "blog_metrics" : {
      "extended_stats" : {
        "field" : "views"
      }
    }
  }
}

After executing the above parameter we would get below response:

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 54,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "blog_metrics": {
      "count": 54,
      "min": 11,
      "max": 1309,
      "avg": 142.85185185185185,
      "sum": 7714,
      "sum_of_squares": 3775934,
      "variance": 49518.05212620027,
      "std_deviation": 222.5265200514318,
      "std_deviation_bounds": {
        "upper": 587.9048919547154,
        "lower": -302.20118825101173
      }
    }
  }
}

In this way, we can fetch these important stats for any field and can get complete insight about the variation in the field value.

Percentile:
Percentile is again a type of metrics aggregations which shows the certain percentage of observed field value at different points. See below example:

GET bqstack/_search?size=0
{
  "aggs": {
    "blog_metrics" : {
      "percentiles" : {
        "field" : "views"
      }
    }
  }
}

After running above expression we should get the following response:

{
"took": 17,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 54,
"max_score": 0,
"hits": []
},
"aggregations": {
"blog_metrics": {
"values": {
"1.0": 11,
"5.0": 12.2,
"25.0": 31,
"50.0": 50.5,
"75.0": 148,
"95.0": 563.7999999999998,
"99.0": 1288.7200000000003
}
}
}
}

There are different other types of metrics aggregations which I am not able to cover in a blog but I have tried to explain all important metrics aggregations

Other Blogs on Elastic Stack:
Introduction to Elasticsearch

Elasticsearch Installation and Configuration on Ubuntu 14.04
Log analysis with Elastic stack 
Elasticsearch Rest API
Basics of Data Search in Elasticsearch
Elasticsearch Rest API
Wildcard and Boolean Search in Elasticsearch
Configure Logstash to push MySQL data into Elasticsearch 
Metrics Aggregation in Elasticsearch
Bucket Aggregation in Elasticsearch
How to create Elasticsearch Cluster

If you found this article interesting, then you can explore “Mastering Kibana 6.0”, “Kibana 7 Quick Start Guide”, “Learning Kibana 7”, and “Elasticsearch 7 Quick Start Guide” books to get more insight about Elastic Stack, how to perform data analysis, and how you can create dashboards for key performance indicators using Kibana.

You can also follow me on:

- LinkedIn: https://www.linkedin.com/in/anubioinfo/

- Twitter: https://twitter.com/anu4udilse

- Medium: https://anubioinfo.medium.com



Comments (0)

Leave a comment

Related Blogs

preview thumbnail
Introduction to Kibana

Aug 1, 2020, 6:19:45 PM | Anurag Srivastava

preview thumbnail
Bucket Aggregation in Elasticsearch

Aug 29, 2018, 7:15:06 PM | Anurag Srivastava

preview thumbnail
Introduction to Elasticsearch Aggregations

Aug 14, 2018, 4:47:56 PM | Anurag Srivastava

preview thumbnail
Wildcard and Boolean Search in Elasticsearch

Aug 10, 2018, 7:14:40 PM | Anurag Srivastava

preview thumbnail
Basics of Data Search in Elasticsearch

Aug 4, 2018, 7:02:21 AM | Anurag Srivastava

preview thumbnail
Elasticsearch REST APIs

Jul 31, 2018, 6:16:42 PM | Anurag Srivastava

Top Blogs

preview thumbnail
Wildcard and Boolean Search in Elasticsearch

Aug 10, 2018, 7:14:40 PM | Anurag Srivastava

preview thumbnail
Elasticsearch REST APIs

Jul 31, 2018, 6:16:42 PM | Anurag Srivastava

preview thumbnail
preview thumbnail
Create a Chess board in PHP

Mar 9, 2020, 8:45:41 AM | Rocky Paul

preview thumbnail
Bucket Aggregation in Elasticsearch

Aug 29, 2018, 7:15:06 PM | Anurag Srivastava

preview thumbnail
Metrics Aggregations in Elasticsearch

Aug 18, 2018, 6:02:20 PM | Anurag Srivastava