Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(transform): Add Send for AWS Lambda #153

Merged
merged 2 commits into from
Apr 3, 2024
Merged

Conversation

jshlbrd
Copy link
Contributor

@jshlbrd jshlbrd commented Apr 1, 2024

Description

  • Adds a send_aws_lambda transform
  • Adds an example of using the new transform to continuously retry a failed S3 trigger

Motivation and Context

This resolves #57 with an emphasis on adding as little new code as possible. It's mostly trivial to build a retry pipeline using existing Substation components. You can test the new example by running this and then uploading a file to the created S3 bucket:

make check
make build-aws
make deploy-aws DEPLOYMENT_DIR=examples/terraform/aws/s3/retry_on_failure AWS_APPCONFIG_ENV=example

How Has This Been Tested?

End-to-end tested using the added example.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@jshlbrd jshlbrd marked this pull request as ready for review April 1, 2024 20:09
@jshlbrd jshlbrd requested a review from a team as a code owner April 1, 2024 20:09
Copy link
Contributor

@shellcromancer shellcromancer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, verified that this implementation meets the needs of an external user as well. 👍

// errSendLambdaPayloadSizeLimit is returned when data exceeds the Lambda
// payload size limit. If this error occurs, then conditions or transforms
// should be applied to either drop or reduce the size of the data.
var errSendLambdaPayloadSizeLimit = fmt.Errorf("data exceeded size limit")
Copy link
Contributor

@shellcromancer shellcromancer Apr 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style: without string formatting or error wrapping we could use errors.New here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since every error gets wrapped in a string to identify the transform (e.g. fmt.Errorf("transform: send_aws_lambda: %v", err)), I'm going to leave this as-is for now for readability consistency.

@jshlbrd jshlbrd merged commit c517ea5 into main Apr 3, 2024
6 checks passed
@jshlbrd jshlbrd deleted the jshlbrd/feat/send-lambda branch April 3, 2024 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automated Failure Recovery for Asynchronous Consumers
2 participants