feat: New batch processing option: 'ThrowOnFullBatchFailure'

[lachriz-aws]

Note

I still have to update the docs, which is why this PR is created as draft.

--

Added new batch processing option: ThrowOnFullBatchFailure to control if a BatchProcessingException should be raised on full batch failure or not.

Please provide the issue number

Issue number: #608

Summary

Changes

Please provide a summary of what's being changed

This PR introduces a new batch processing option: ThrowOnFullBatchFailure to control if a BatchProcessingException should be raised on full batch failure or not.

The processing option can be set via:

• A new ThrowOnFullBatchFailure field on the BatchProcessor attribute (applied in non-utility mode).
• A new environment variable POWERTOOLS_BATCH_THROW_ON_FULL_BATCH_FAILURE (applied in non-utility mode).
• Directly on the AWS.Lambda.Powertools.BatchProcessing.ProcessingOptions in utility mode.

To maintain current behavior, the new setting defaults to true, but as pointed out on a similar feature request in the Powertools for AWS Lambda (TypeScript) repo, here: aws-powertools/powertools-lambda-typescript#2122 we might want to change that for the next major version release.

To maintain feature parity, the new setting has been named similar to how it has been implemented in Powertools for AWS Lambda (TypeScript), see: https://github.com/arnabrahman/aws-lambda-powertools-typescript/blob/main/packages/batch/src/types.ts#L32

User experience

Please share what the user experience looks like before and after this change

Imagine a scenario where processing of an entire batch has failed: if ThrowOnFullBatchFailure is set to false then the Batch Processor will not throw an exception after batch processing has completed. Instead, it will simply return the ids of the items that failed processing (all batch items in this case) and exit gracefully.

In the scenario that a Lambda function is configured with ErrorHandlingPolicy = StopOnFirstBatchItemFailure and the first batch item fails processing, then the entire batch is marked as failed (as per the docs: https://docs.powertools.aws.dev/lambda/dotnet/utilities/batch-processing/#error-handling-policy). In this case, if ThrowOnFullBatchFailure is set to false, then the same behavior as outlined above will apply (in other words, the new ThrowOnFullBatchFailure option is compatible with the different error handling policies we have today).

Checklist

Please leave checklist items unchecked if they do not apply to your change.

• Meets tenets criteria
• I have performed a self-review of this change
• Changes have been tested
• Changes are documented
• PR title follows conventional commit semantics

Is this a breaking change? No

RFC issue number:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[hjgraca]

@lachriz-aws thank you for opening the pull request. There are two small issues raised by SonarCloud. https://sonarcloud.io/project/issues?sinceLeakPeriod=true&issueStatuses=OPEN%2CCONFIRMED&pullRequest=646&id=aws-powertools_powertools-lambda-dotnet&open=AZILxhrgZTYLizOkweVb. Can you address them please? thank you

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.51%. Comparing base (5a483b7) to head (e1d88c2).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #646      +/-   ##
===========================================
- Coverage    72.71%   72.51%   -0.20%     
===========================================
  Files          190      190              
  Lines         7886     7893       +7     
  Branches       850      850              
===========================================
- Hits          5734     5724      -10     
- Misses        1862     1875      +13     
- Partials       290      294       +4     

Flag	Coverage Δ
unittests	72.51% <100.00%> (-0.20%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[lachriz-aws]

@lachriz-aws thank you for opening the pull request. There are two small issues raised by SonarCloud. https://sonarcloud.io/project/issues?sinceLeakPeriod=true&issueStatuses=OPEN%2CCONFIRMED&pullRequest=646&id=aws-powertools_powertools-lambda-dotnet&open=AZILxhrgZTYLizOkweVb. Can you address them please? thank you

Yes, absolutely. Now done.

When time allows, please let me know what you think of the proposed solution and I will be happy to get going updating the docs as well. Thank you :)