Incorrect report for CIS 3.5 (AWS Config)

[bwhaley]

Describe the bug
The CIS 3.5 control requires that only one region has includeGlobalResourceTypes enabled. However, this check appears to fail unless includeGlobalResourceTypes is true in every region.

Steampipe version (steampipe -v)
0.5.0

Plugin version (steampipe plugin list)
hub.steampipe.io/plugins/turbot/aws@latest | 0.18.0 | aws

To reproduce

1. Enable AWS Config in every region of the account and ensure that recording is enabled and status is successful.
2. In exactly one region, ensure includeGlobalResourceTypes is true.
3. Run steampipe check benchmark.cis_v130
4. Note that the output shows ALARM state for every region except the region where includeGlobalResourceTypes is true.

ALARM: eu-north-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ................................................ eu-north-1 000476352289
ALARM: us-east-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. .................................................. us-east-1 000476352289
ALARM: me-south-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ................................................ me-south-1 000476352289
ALARM: ap-northeast-2 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ........................................ ap-northeast-2 000476352289
ALARM: ca-central-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ............................................ ca-central-1 000476352289
ALARM: us-west-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. .................................................. us-west-1 000476352289
OK   : us-west-2 IncludeGlobalResourceTypes enabled, AllSupported enabled, Recording enabled and LastStatus is SUCCESS. ......................................................... us-west-2 000476352289
ALARM: ap-northeast-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ........................................ ap-northeast-1 000476352289
ALARM: ap-east-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. .................................................. ap-east-1 000476352289
ALARM: ap-southeast-2 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ........................................ ap-southeast-2 000476352289
ALARM: us-east-2 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. .................................................. us-east-2 000476352289
ALARM: eu-west-2 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. .................................................. eu-west-2 000476352289
ALARM: ap-southeast-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ........................................ ap-southeast-1 000476352289
ALARM: sa-east-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. .................................................. sa-east-1 000476352289
ALARM: eu-south-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ................................................ eu-south-1 000476352289
ALARM: eu-central-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ............................................ eu-central-1 000476352289
ALARM: ap-northeast-3 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ........................................ ap-northeast-3 000476352289
ALARM: eu-west-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. .................................................. eu-west-1 000476352289
ALARM: ap-south-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ................................................ ap-south-1 000476352289
ALARM: af-south-1 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. ................................................ af-south-1 000476352289
ALARM: eu-west-3 IncludeGlobalResourceTypes disabled, AllSupported disabled, Recording disabled and LastStatus is not SUCCESS. .................................................. eu-west-3 000476352289

Expected behavior
As long as one region has IncludeGlobalResourceTypes enabled, all regions should report OK.

[cbruno10]

@bwhaley Thanks for the report!

Our query does look incorrect and should only alarm for the account if no region has IncludeGlobalResourceTypes enabled.

I also believe we have another issue for this query which could prevent it from returning the correct result (#27), so we'll look into both of these issues and see how we can get the control working properly.

Related to the other potential bug above, can you please paste here what regions you have enabled in your .steampipe/config/aws.spc file? Thanks!

[rajlearner17]

Reported here

[rhoboat]

Adding onto this, when I added 22 regions to the ~/.steampipe/config/aws.spc file as below:

  regions = [
    "us-east-1",
    "us-east-2",
    "us-west-1",
    "us-west-2",
    "eu-west-1",
    "eu-west-2",
    "eu-west-3",
    "eu-south-1",
    "eu-central-1",
    "eu-north-1",
    "ap-south-1",
    "ap-northeast-1",
    "me-south-1",
    "ap-east-1",
    "ap-southeast-3",
    "ap-northeast-3",
    "ap-southeast-1",
    "af-south-1",
    "sa-east-1",
    "ap-southeast-2",
    "ap-northeast-2",
    "ca-central-1"
  ]

I got the following result:

As you can see, there are 22-16 = 6 regions where the check fails. In this particular AWS account, I can confirm that we have AWS Config disabled for those 6 regions.

So I'm not sure this is a bug, actually.

[rajlearner17]

@rhoboat Appreciate checking this out. The result you have mentioned is correct for the use case you have tested. We have already addressed this issue in detail here. I guess it's time to close this one with a good test result from you 😄

Once again thank you 👍 for responding. Pls keep us posted, in case you come across any other observations