[Bug]: CloudWatch resources can no longer be refreshed with default ReadOnlyAccess policy

[stevehipwell]

Terraform Core Version

1.3.6

AWS Provider Version

4.47.0

Affected Resource(s)

Observed:

• aws_cloudwatch_log_group

Expected:

• aws_cloudwatch_*

Expected Behavior

I'd expect to be able to plan and refresh state using the default AWS ReadOnlyAccess policy (arn:aws:iam::aws:policy/ReadOnlyAccess).

Actual Behavior

The state couldn't be refreshed due to the logs:ListTagsForResource action not being present in the default AWS ReadOnlyAccess policy (arn:aws:iam::aws:policy/ReadOnlyAccess).

Relevant Error/Panic Output Snippet

Error: listing tags for CloudWatch Logs Log Group (/aws/eks/eks-8/cluster): AccessDeniedException: User: arn:aws:sts::0000000000000:assumed-role/Terraform/vault-github-Pull_Request-eks-1671188089-0cv6 is not authorized to perform: logs:ListTagsForResource on resource: arn:aws:logs:eu-west-1:0000000000000:log-group:/aws/eks/eks-8/cluster because no session policy allows the logs:ListTagsForResource action
	status code: 400, request id: 5d53270e-1a1f-46fa-91b6-2f068940009a

  with module.eks.module.control_plane.aws_cloudwatch_log_group.default,
  on .terraform/modules/eks/modules/control-plane/logging.tf line 1, in resource "aws_cloudwatch_log_group" "default":
   1: resource "aws_cloudwatch_log_group" "default" {

Terraform Configuration Files

resource "aws_cloudwatch_log_group" "default" {
  name              = "/aws/eks/${var.name}/cluster"
  retention_in_days = var.control_plane_log_retention
  kms_key_id        = null

  tags = var.tags
}

Steps to Reproduce

• Create the resource above using an AWS role with write permissions
• Make a change and run a plan using an AWS role with only the default AWS ReadOnlyAccess policy (arn:aws:iam::aws:policy/ReadOnlyAccess)

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

• [Enhancement]: Use Correct API's for CloudWatch Logs Tagging #28322
• CloudWatch Logs: Use TagResource API #28359

Would you like to implement a fix?

No

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[jungseoklee]

Could you attach arn:aws:iam::aws:policy/CloudWatchLogsReadOnlyAccess to IAM role you use as mitigation if you want to keep AWS Provider Version 4.47.0? Otherwise, I think you need to wait till logs:ListTagsForResource is part of arn:aws:iam::aws:policy/ReadOnlyAccess.

[stevehipwell]

@jungseoklee we use the R/O policy ARN for a large number of roles, this is a major breaking change with a high blast radius.

The point here is that if the changes were made knowing the impact then they should have been SemVer major with documentation changes. If the impact was unknown then this is a defect and should be reverted until it can be delivered to spec.

[jungseoklee]

@stevehipwell I agree with your point. I just want to share workaround as user.

[n3mawashi]

Hi, Came across this, this morning. This is my work around for now.

inline policy for the role.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "logsListTagsForResource", "Effect": "Allow", "Action": "logs:ListTagsForResource", "Resource": "*" } ] }

[ewbankkit]

@stevehipwell Thanks for raising this issue 👏 and apologies that the change of tagging APIs caused problems.
We have reverted aws_cloudwatch_log_group to use the log group-specific APIs via #28492, which will be included in v4.49.0 of the Terraform AWS Provider, likely to be released in the first week of the New Year (January 5th 2023).

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.