Cherry-pick #17658 to 7.x: Add support for IAM role arn in aws config

[kaiyan-sheng]

Cherry-pick of PR #17658 to 7.x branch. Original message:

What does this PR do?

This PR is to add support for IAM role arn in AWS credentials config.

Why is it important?

When user doesn't want to store any credentials for Metricbeat/Filebeat locally(for example in EC2 instance), it's better to leverage AWS IAM role. A role does not have standard long-term credentials such as a password or access keys associated with it. Instead, when you assume a role, it provides you with temporary security credentials for your role session. IAM role Amazon Resource Name (ARN) can be used to specify which AWS IAM role to assume to generate temporary credentials.

Using role_arn also solves reload temporary credential problem in #17189. sts.NewAssumeRoleProvider with role_arn input constructs and returns a credentials provider that will retrieve credentials by assuming a IAM role using STS. AssumeRoleProvider has a a function retrieveFn, which generates a new set of temporary credentials using STS.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

1. Create IAM role in AWS console https://console.aws.amazon.com/iam/home#/roles
2. Attach IAM role with a policy like below to give permissions needed for testing Metricbeat and Filebeat:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "sqs:ReceiveMessage"
            ],
            "Resource": [
                "arn:aws:s3:::test-fb-ks/*",
                "arn:aws:sqs:us-east-1:428152502467:test-fb-ks"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "sqs:ChangeMessageVisibility",
            "Resource": "arn:aws:sqs:us-east-1:428152502467:test-fb-ks"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "sqs:DeleteMessage",
            "Resource": "arn:aws:sqs:us-east-1:428152502467:test-fb-ks"
        },
        {
            "Sid": "VisualEditor3",
            "Effect": "Allow",
            "Action": [
                "sqs:ListQueues",
                "tag:GetResources",
                "ec2:DescribeInstances",
                "cloudwatch:GetMetricData",
                "ec2:DescribeRegions",
                "iam:ListAccountAliases",
                "sts:GetCallerIdentity",
                "cloudwatch:ListMetrics"
            ],
            "Resource": "*"
        }
    ]
}

3. Copy the role ARN, for example: arn:aws:iam::428152502467:role/test-mb
4. Enable Metricbeat aws module:

./metricbeat modules enable aws

5. Change modules.d/aws.yml to use role_arn:

- module: aws
  role_arn: arn:aws:iam::428152502467:role/test-mb
  period: 5m
  metricsets:
    - ec2

6. Start Metricbeat with ./metricbeat -e and with the correct permissions in IAM role, ec2 metrics should be collected and sent to Elasticsearch.

Related issues

• Closes Please support AWS IAM Instance profiles! #12464

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)