Is it possible to use AWS Account ID and Region in a filter?

[CallumHibbert]

I would like to use the AWS Account ID and/or Region in a filter.

I am building some alerts on CloudWatch metrics but the resources returned by CloudCustodian don't contain those identififers, there's no ARN because I assume CloudWatch metrics don't have an ARN.

Example resources:

[
  {
    "filterName": "my-filter-1",
    "filterPattern": "...",
    "metricTransformations": [
      {
        "metricName": "LambdaBilledDuration",
        "metricNamespace": "my-namespace-1",
        "metricValue": "$billed_duration_value",
        "unit": "None"
      }
    ],
    "creationTime": ...,
    "logGroupName": "/aws/lambda/my-log-group-1",
    "c7n:MatchedFilters": [
      "filterName"
    ]
  },
  {
    "filterName": "my-filter-2",
    "filterPattern": "...",
    "metricTransformations": [
      {
        "metricName": "LambdaBilledDuration",
        "metricNamespace": "my-namespace-2",
        "metricValue": "$billed_duration_value",
        "unit": "None"
      }
    ],
    "creationTime": ...,
    "logGroupName": "/aws/lambda/my-log-group-2",
    "c7n:MatchedFilters": [
      "filterName"
    ]
  },
  ...
]

With the above resource information there is no way to filter on AWS Account ID and Region. With other resources (e.g. Lambda, S3, DynamoDB etc) those contain an ARN which allows for unique identification across accounts and regions via the ARN.

I don't think the conditions feature (https://cloudcustodian.io/docs/quickstart/advanced.html#conditional-policy-execution) helps because a condition disables the policy for an account or region.

I would want to do something like:

  filters:
  - type: value
    key: {account_id}-{region}-filterName
    op: not-in
    value_from:
        ...

(I am aware the pseudo code above for the key is incorrect).

Is there a way to access the overall policy metadata (account ID, region etc) from within a filter?

[ajkerrigan]

Sure, you can do that. If you were looking for a filter named my-filter-1 in us-east-2 of account 111111111111, you could use the key of your value filter to mush those elements into a single string. Your pseudocode was headed in the right direction and pretty close! A couple variations on an approach to do that:

key: join('-', ['{account_id}', '{region}', filterName])
key: join('', ['{account_id}-{region}-', filterName])

And then you could check your matches against a list of known account/region/filterName combinations:

filters:
  - type: value
    key: join('-', ['{account_id}', '{region}', filterName])
    op: in
    value:
      - '111111111111-us-east-1-my-filter-1'
      - '111111111111-us-east-2-my-filter-1'
      - '111111111111-us-east-2-my-filter-2'
      - '222222222222-us-west-2-my-filter-62'

...and from there, use value_from as in your original example to maintain your list of known values in S3.

[CallumHibbert]

Perfect answer, tested and working, thanks.

[ajkerrigan]

Glad to hear it! I should say that in your case, one other option that might be relevant is having a JSON file structured hierarchically. For example if you had:

s3://my_bucket/log_filters.json

{
    "111111111111": {
        "us-east-1": [
            "my-filter-1"
        ],
        "us-east-2": [
            "my-filter-1",
            "my-filter-2"
        ]
    },
    "222222222222": {
        "us-west-2": [
            "my-filter-62"
        ]
    }
}

Then you could have a filter that looked filters up by account/region pair at runtime:

filters:
  - type: value
    key: filterName
    op: in
    value_from:
      url: s3://my_bucket/log_filters.json
      expr: '"{account_id}"."{region}"[]'

In your case it sounds like dash-joining may be simpler and more straightforward, but this alternative is handy sometimes too and worth being aware of.