[Metricbeat]Combine metrics with no dimension into one event

[kaiyan-sheng]

What does this PR do?

When collecting CloudWatch metrics with no dimension, they should be reported in the same metric/event instead of separate metrics/events. For example, lambda namespace reports metrics across all functions: ConcurrentExecutions, Duration, Errors, Invocations, Throttles. For metrics across all functions, they represent aggregate metrics for all functions in the current AWS Region. Instead of separating these metrics into individual events, cloudwatch metricset should report them as one metric.

Why is it important?

Combining metrics with no dimension into one event will easily show users all the aggregated metrics from CloudWatch in one event.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

1. make sure there is at least one lambda function and also check cloudwatch to see if there are metrics for that lambda function.
2. enable aws module by ./metricbeat modules enable aws
3. change aws.yml under modules.d to:

- module: aws
  period: 30s
  credential_profile_name: elastic-beats
  metricsets:
    - lambda

4. You should see events in Kibana with/without dimensions. There should be only one event without dimension which includes all CloudWatch aggregated metrics. For example:

{
  "_index": "metricbeat-8.0.0-2020.03.09-000001",
  "_type": "_doc",
  "_id": "YMxZLXEBCsIwLMtCN8hn",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2020-03-30T21:29:02.851Z",
    "ecs": {
      "version": "1.5.0"
    },
    "host": {
      "os": {
        "family": "darwin",
        "name": "Mac OS X",
        "kernel": "17.7.0",
        "build": "17G10021",
        "platform": "darwin",
        "version": "10.13.6"
      },
    "cloud": {
      "provider": "aws",
      "region": "us-west-1",
      "account": {
        "name": "elastic-beats",
        "id": "428152502467"
      }
    },
    "aws": {
      "lambda": {
        "metrics": {
          "Throttles": {
            "avg": 0
          },
          "Duration": {
            "avg": 3003.1659999999997
          },
          "Errors": {
            "avg": 1
          },
          "ConcurrentExecutions": {
            "avg": 1
          },
          "Invocations": {
            "avg": 1
          }
        }
      },
      "cloudwatch": {
        "namespace": "AWS/Lambda"
      }
    },
    "event": {
      "duration": 9918983165,
      "dataset": "aws.lambda",
      "module": "aws"
    },
    "metricset": {
      "name": "lambda",
      "period": 300000
    },
    "service": {
      "type": "aws"
    }
  },
  "fields": {
    "@timestamp": [
      "2020-03-30T21:29:02.851Z"
    ]
  },
  "sort": [
    1585603742851
  ]
}

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[ChrsMark]

lgtm! Wondering if we could have a unit test to ensure this functionality.

[kaiyan-sheng]

@ChrsMark Totally agree on adding more unit test. I just added two tests for CreateEvents function. Thank you!