feat(logger): add logger buffer feature

[leandrodamascena]

🚨 Status

Under Construction: This PR is a work in progress

Issue number: #4432

Summary

Changes

This pull request introduces a new logging buffer mechanism to optimize CloudWatch Logs cost and provide better debugging context during exceptions. Implement a smart log buffering system for AWS Lambda that captures debug context only when exceptions occur, reducing CloudWatch Logs costs.

The logging buffer will utilize a ring buffer implementation to manage memory efficiently, with a configurable buffer size that allows developers to control memory allocation and log retention. The mechanism will be designed with minimal performance overhead, ensuring that the log buffering process does not introduce significant computational cost or latency.

Checklist

Log Buffering

• Add new parameter to logger constructor
• Add new method to buffer log entries
• Modify existing log methods to support buffering configuration
• Ensure log line location tracking
• Ensure precise timestamp tracking
• Add method to compress and uncompress

Error Handling

• Implement decorator for exceptions
• Validate flush logic based on configuration flags

Testing

• Write functional tests
• Write e2e tests
• Write unit tests

Documentation

• Improve documentation
• Add detailed usage examples
• Add configuration guide
• Add performance considerations

User experience

from aws_lambda_powertools import Logger
from aws_lambda_powertools.logging.buffer import LoggerBufferConfig
from aws_lambda_powertools.utilities.typing import LambdaContext

logger_buffer = LoggerBufferConfig(max_size=10240, minimum_log_level="DEBUG")
logger = Logger(logger_buffer=logger_buffer)

def lambda_handler(event: dict, context: LambdaContext) -> dict:
    logger.debug("This will be buffered")
    return {"ok"}

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• Meet tenets criteria
• I have performed a self-review of this change
• Changes have been tested
• Changes are documented
• PR title follows conventional commit semantics

Is this a breaking change?

RFC issue number:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[codecov]

Codecov Report

Attention: Patch coverage is 94.57364% with 7 lines in your changes missing coverage. Please review.

Project coverage is 96.24%. Comparing base (ccaf426) to head (e39229a).
Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
aws_lambda_powertools/logging/logger.py	91.11%	2 Missing and 2 partials ⚠️
aws_lambda_powertools/logging/types.py	0.00%	2 Missing ⚠️
aws_lambda_powertools/logging/buffer/cache.py	96.87%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #6060      +/-   ##
===========================================
- Coverage    96.25%   96.24%   -0.02%     
===========================================
  Files          234      238       +4     
  Lines        11140    11256     +116     
  Branches       822      844      +22     
===========================================
+ Hits         10723    10833     +110     
- Misses         327      330       +3     
- Partials        90       93       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud