Skip to content

Commit

Permalink
fix(forwarder): filter eventbridge triggers by source bucket (#307)
Browse files Browse the repository at this point in the history
We previously triggered the forwarder lambda on all S3 object created
events, and relied on the lambda to ignore spurious triggers. This turns
out to be too lax. We can filter events by source bucket using the
`wildcard` pattern. Like bucket names, the wildcard pattern supports
globs. Without globs, it behaves as an exact match. This means that for
source bucket names `foo*` and `bar`, the event bridge pattern compiled
would be:

```
{
  "source": ["aws.s3"],
  "detail-type": ["Object Created"],
  "detail.bucket.name": [{
      "wildcard": "foo*"
    },
    {
      "wildcard": "bar"
    }
  ]
}
```
  • Loading branch information
jta authored Jun 17, 2024
1 parent 23bf023 commit 6ae79b0
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 12 deletions.
22 changes: 13 additions & 9 deletions apps/forwarder/template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,8 @@ Conditions:
- ''
- !Ref SourceBucketNames
- ''
EnableSourceS3: !Not
- !Condition DisableSourceS3
DisableKMSDecrypt: !Equals
- !Join
- ''
Expand Down Expand Up @@ -234,17 +236,19 @@ Resources:
- !Ref Queue
Rule:
Type: AWS::Events::Rule
Condition: EnableSourceS3
Properties:
Description: "Trigger copy for object created events"
EventPattern:
source:
- "aws.s3"
detail-type:
- "Object Created"
# NOTE: it would be nice to filter events to match source buckets only.
# SourceBucketArns however allows for wildcards, which aren't easily
# converted into the appropriate EventBridge filter. We instead filter
# within the lambda.
EventPattern: !Sub
- |
{
"source": ["aws.s3"],
"detail-type": ["Object Created"],
"detail.bucket.name": [{"wildcard": "${wildcards}"}]
}
- wildcards: !Join
- '"}, {"wildcard":"'
- !Ref SourceBucketNames
Targets:
- Arn: !GetAtt Queue.Arn
Id: "Forwarder"
Expand Down
7 changes: 4 additions & 3 deletions integration/tests/forwarder.tftest.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -120,9 +120,10 @@ run "install_forwarder" {
setup = run.setup
app = "forwarder"
parameters = {
DataAccessPointArn = run.target_bucket.access_point.arn
DestinationUri = "s3://${run.target_bucket.access_point.alias}/"
SourceBucketNames = "${join(",", [for k, v in run.sources.buckets : v.id])}"
DataAccessPointArn = run.target_bucket.access_point.arn
DestinationUri = "s3://${run.target_bucket.access_point.alias}/"
# all bucket names share the same prefix, this should just work.
SourceBucketNames = "${run.setup.short}*"
SourceTopicArns = "arn:aws:sns:${run.setup.region}:${run.setup.account_id}:*"
ContentTypeOverrides = "${var.override_match}=${var.override_content_type}"
SourceKMSKeyArns = "${join(",", [for k, v in run.sources.buckets : v.kms_key.arn if v.kms_key != null])}"
Expand Down

0 comments on commit 6ae79b0

Please sign in to comment.