feat(batch): add option to not throw FullBatchFailureError when the entire batch fails

[arnabrahman]

Summary

Currently, when using the Batch Processor utility, if all the records in a batch are marked as failed, the utility throws a FullBatchFailureError. However, there can be scenarios where we do not want to throw an error for this condition.

This PR adds a throwOnFullBatchFailure option, giving users the ability to bypass the FullBatchFailureError when the entire batch fails.

Changes

• Introduce throwOnFullBatchFailure option that can be passed as an option inside processPartialResponse or processPartialResponseSync functions
• During full batch failure
  • If throwOnFullBatchFailure is false, we will not throw FullBatchFailureError exception
  • If throwOnFullBatchFailure is not provided or is true, we will keep the current implementation of throwing FullBatchFailureError exception
• Update the relevant doc blocks
• Unit tests for full batch failure for processPartialResponseandprocessPartialResponseSync` function
• I believe we have to update the docs but couldn't figure it out in which section. For now, I have added a line for the option.

Issue number: #2122

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[leandrodamascena]

Note to remember: @dreamorosi please let's talk before/after merging this PR to see if it makes sense to add the same in Python and @aws-powertools/lambda-dotnet-core

cc @hjgraca

[dreamorosi]

Hi @leandrodamascena, we talked about if a few weeks ago when we did the v3 planning, the consensus was yes, since it impacts the scaling of the poller in some cases.

We can talk about it again also.

[dreamorosi]

Hi @arnabrahman - thank you for the PR!

I'll wait until I can talk with @leandrodamascena tomorrow to start reviewing the PR.

Thank you for your patience!

[dreamorosi]

Hi @arnabrahman - thank you for the patience.

I'll make a couple of commits to the branch to explain the feature in the documentation before we can merge it.

[dreamorosi]

Note to remember: @dreamorosi please let's talk before/after merging this PR to see if it makes sense to add the same in Python and @aws-powertools/lambda-dotnet-core

cc @hjgraca

And to follow up on this: we went through the feature and we are aligned on the fact that we should move forward. We will also add the same flag on the other versions as time allows it.

[leandrodamascena]

Approved! Amazing work @arnabrahman and @dreamorosi!

I'm adding this new feature in Python.