Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(s3): add skip destination validation property #30916

Merged
merged 13 commits into from
Aug 19, 2024

Conversation

yerzhan7
Copy link
Contributor

@yerzhan7 yerzhan7 commented Jul 22, 2024

Issue #30914

Closes #30914.

Reason for this change

When customers call this API to setup S3 notification configuration for SQS/SNS/Lambda S3 sends s3:TestEvent in order to validate permissions. (For Lambda it does dryrun function invocation instead)

However, some customers do not want to do that and test permissions during CDK deployment.

Internal reference: 49359101-0e5e-43f3-99eb-3c6c5ed68db1

For example, one customer does not want these test events because they have alarm on unconsumed messages in SQS and they do not have any SQS consumers. And they update notification configuration frequently, which leads to many test events in the queue. See internal ticket: P142186522

Description of changes

Expose skip destination validation property when calling PutBucketNotification API in Bucket props.

Description of how you validated changes

Unit test updated.

Updated integration tests. Note that 2 integration tests I had to fix and run them with --disable-update-workflow flag because they were failing:

  • integ.s3.imported-bucket.js test failed because someone already created bucket cdk-integration-test-s3-imported-bucket-name
  • integ.bucket-notifications.js test failed because of overlapping suffix error (not sure how it was passing previously):
❌  cdk-integ-lambda-bucket-s3-notifications failed: Error: The stack named cdk-integ-lambda-bucket-s3-notifications failed to deploy: UPDATE_FAILED 
(The following resource(s) failed to update: [Construct1IntegUnmanagedBucket1Notifications4A1599D7]. ): Received response status [FAILED] from custom resource. 
Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. 
Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. 
See the details in CloudWatch Log Stream: 2024/07/22/[$LATEST]e6a16cf979dd4671998e7d911769ff42 (RequestId: 19f6fcd7-d31d-4fbf-9f4a-e3b7cba1cd2b), 
Rolling back the failed resource only., Received response status [FAILED] from custom resource. Message returned: Error: An error occurred 
(InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. 
Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. 
See the details in CloudWatch Log Stream: 2024/07/22/[$LATEST]c30efd0375d64b8088e0ee64d63ce4db (RequestId: 19f6fcd7-d31d-4fbf-9f4a-e3b7cba1cd2b)

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions bot added the valued-contributor [Pilot] contributed between 6-12 PRs to the CDK label Jul 22, 2024
@aws-cdk-automation aws-cdk-automation requested a review from a team July 22, 2024 09:52
@github-actions github-actions bot added feature-request A feature should be added or improved. p2 labels Jul 22, 2024
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@github-actions github-actions bot added the effort/medium Medium work item – several days of effort label Jul 22, 2024
@aws-cdk-automation aws-cdk-automation dismissed their stale review July 22, 2024 22:14

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Jul 22, 2024
Copy link
Contributor

@gracelu0 gracelu0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @yerzhan7 , thanks for adding support for this feature! I have some questions about the runtime update before approving but most of the changes LGTM.

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 9, 2024
Co-authored-by: Grace Luo <54298030+gracelu0@users.noreply.github.com>
@mergify mergify bot dismissed gracelu0’s stale review August 9, 2024 13:15

Pull request has been modified.

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 9, 2024
@gracelu0
Copy link
Contributor

gracelu0 commented Aug 14, 2024

Just checking the integration test cases that were updated before approving. I was able to run integ.s3.imported-bucket.js and integ.bucket-notifications.js successfully locally off the main branch.

Ah okay was able to reproduce the imported bucket error, I think that change makes sense as we shouldn't be hardcoding the bucket name.

Was also able to reproduce the overlapping prefixes/suffixes error, also not sure how it was passing before. Thanks for the updates!

gracelu0
gracelu0 previously approved these changes Aug 14, 2024
Copy link
Contributor

@gracelu0 gracelu0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution!

Copy link
Contributor

mergify bot commented Aug 14, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 14, 2024
Copy link
Contributor

mergify bot commented Aug 14, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot dismissed gracelu0’s stale review August 14, 2024 18:08

Pull request has been modified.

gracelu0
gracelu0 previously approved these changes Aug 14, 2024
Copy link
Contributor

mergify bot commented Aug 14, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot dismissed gracelu0’s stale review August 14, 2024 19:20

Pull request has been modified.

@gracelu0
Copy link
Contributor

@Mergifyio update

Copy link
Contributor

mergify bot commented Aug 19, 2024

update

❌ Mergify doesn't have permission to update

For security reasons, Mergify can't update this pull request. Try updating locally.
GitHub response: refusing to allow a GitHub App to create or update workflow .github/workflows/request-cli-integ-test.yml without workflows permission

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 19, 2024
@gracelu0
Copy link
Contributor

@Mergifyio update

Copy link
Contributor

mergify bot commented Aug 19, 2024

update

✅ Branch has been successfully updated

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 26b21e1
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Copy link
Contributor

mergify bot commented Aug 19, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit ffc40e5 into aws:main Aug 19, 2024
10 checks passed
Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 19, 2024
@yerzhan7 yerzhan7 deleted the s3-skip-notification-destination-validation branch August 19, 2024 19:40
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2 pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. valued-contributor [Pilot] contributed between 6-12 PRs to the CDK
Projects
None yet
Development

Successfully merging this pull request may close these issues.

aws-s3: Expose SkipDestinationValidation option when setting bucket notifications
3 participants