v2.3.0

[heitorlessa]

Summary

This release features: (1) Extract CloudWatch Logs sent to Kinesis streams, (2) Handling multiple exceptions in Event Handler REST, and (3) Log uncaught exceptions in Logger.

Big thanks to new contributors: @ascopes (logger static typing), @bnsouza (expand testing docs in Event Handler), @kt-hr (bugfix multiple routes in Event Handler), @mangoes-git (bugfix for fetching SecretBinary)

Extracting CloudWatch Logs shipped to Kinesis Data Streams

Docs

This address a common use case of centralizing CloudWatch Logs from multiple regions or accounts using Kinesis Data Streams.

You can now easily extract CloudWatch Logs (decompress, decode, etc.) using either Event Source Data Classes or Parser. It also seamless integrates with Batch so you can benefit from partial failure handling among other benefits, when processing large batch of logs from Kinesis.

Event Source Data Classes

from aws_lambda_powertools.utilities.batch import (BatchProcessor, EventType,
                                                   batch_processor)
from aws_lambda_powertools.utilities.data_classes.kinesis_stream_event import (
    KinesisStreamRecord, extract_cloudwatch_logs_from_record)

processor = BatchProcessor(event_type=EventType.KinesisDataStreams)


def record_handler(record: KinesisStreamRecord):
    log = extract_cloudwatch_logs_from_record(record)
    return log.message_type == "DATA_MESSAGE"


@batch_processor(record_handler=record_handler, processor=processor)
def lambda_handler(event, context):
    return processor.response()

Parser

from aws_lambda_powertools.utilities.batch import (BatchProcessor, EventType,
                                                   batch_processor)
from aws_lambda_powertools.utilities.parser.models import (
    KinesisDataStreamModel, KinesisDataStreamRecord)
from aws_lambda_powertools.utilities.parser.models.kinesis import \
    extract_cloudwatch_logs_from_record

processor = BatchProcessor(event_type=EventType.KinesisDataStreams, model=KinesisDataStreamModel)


def record_handler(record: KinesisDataStreamRecord):
    log = extract_cloudwatch_logs_from_record(record)
    return log.messageType == "DATA_MESSAGE"


@batch_processor(record_handler=record_handler, processor=processor)
def lambda_handler(event, context):
    return processor.response()

Handling multiple exceptions in Event Handler

Docs

You can now catch multiple exceptions when registering an exception handler.

This is useful when you have related exceptions you want to handle the same way.

import requests

from aws_lambda_powertools.event_handler import (
    APIGatewayRestResolver,
    Response,
    content_types,
)
from aws_lambda_powertools.utilities.typing import LambdaContext

app = APIGatewayRestResolver()


@app.exception_handler([ValueError, requests.HTTPError])
def handle_invalid_limit_qs(ex: ValueError | requests.HTTPError):  # receives exception raised
    metadata = {"path": app.current_event.path, "query_strings": app.current_event.query_string_parameters}
    logger.error(f"Malformed request: {ex}", extra=metadata)

    return Response(
        status_code=400,
        content_type=content_types.TEXT_PLAIN,
        body="Invalid request parameters.",
    )


@app.get("/todos")
def get_todos():
    max_results: int = int(app.current_event.get_query_string_value(name="limit", default_value=0))

    todos: requests.Response = requests.get(f"https://jsonplaceholder.typicode.com/todos?limit={max_results}")
    todos.raise_for_status()

    return {"todos": todos.json()}


def lambda_handler(event: dict, context: LambdaContext) -> dict:
    return app.resolve(event, context)

Logging uncaught exceptions

Docs

You can now automatically log uncaught exceptions using Logger via log_uncaught_exceptions=True.

Logger will use Python's exception hook to log such exception with your pre-configured Logger instance for maximum context.

import requests

from aws_lambda_powertools import Logger
from aws_lambda_powertools.utilities.typing import LambdaContext

ENDPOINT = "http://httpbin.org/status/500"
logger = Logger(log_uncaught_exceptions=True)


def handler(event: dict, context: LambdaContext) -> str:
    ret = requests.get(ENDPOINT)
    # HTTP 4xx/5xx status will lead to requests.HTTPError
    # Logger will log this exception before this program exits non-successfully
    ret.raise_for_status()

    return "hello world"

This would generate the following output in CloudWatch Logs

{
    "level": "ERROR",
    "location": "log_uncaught_exception_hook:756",
    "message": "500 Server Error: INTERNAL SERVER ERROR for url: http://httpbin.org/status/500",
    "timestamp": "2022-11-16 13:51:29,198+0100",
    "service": "payment",
    "exception": "Traceback (most recent call last):\n  File \"<input>\", line 52, in <module>\n    handler({}, {})\n  File \"<input>\", line 17, in handler\n    ret.raise_for_status()\n  File \"<input>/lib/python3.9/site-packages/requests/models.py\", line 1021, in raise_for_status\n    raise HTTPError(http_error_msg, response=self)\nrequests.exceptions.HTTPError: 500 Server Error: INTERNAL SERVER ERROR for url: http://httpbin.org/status/500",
    "exception_name": "HTTPError"
}

Changes

🌟New features and non-breaking changes

• feat(parser): export Pydantic.errors through escape hatch (feat(parser): export Pydantic.errors through escape hatch #1728) by @heitorlessa
• feat(logger): log uncaught exceptions via system's exception hook (feat(logger): log uncaught exceptions via system's exception hook #1727) by @heitorlessa
• feat(parser): extract CloudWatch Logs in Kinesis streams (feat(parser): extract CloudWatch Logs in Kinesis streams #1726) by @heitorlessa
• feat(event_sources): extract CloudWatch Logs in Kinesis streams (feat(event_sources): extract CloudWatch Logs in Kinesis streams #1710) by @heitorlessa
• feat(apigateway): multiple exceptions in exception_handler (feat(apigateway): multiple exceptions in exception_handler #1707) by @sprkem

📜 Documentation updates

• docs(idempotency): add missing Lambda Context; note on thread-safe (docs(idempotency): add missing Lambda Context; note on thread-safe #1732) by @heitorlessa
• feat(logger): log uncaught exceptions via system's exception hook (feat(logger): log uncaught exceptions via system's exception hook #1727) by @heitorlessa
• feat(event_sources): extract CloudWatch Logs in Kinesis streams (feat(event_sources): extract CloudWatch Logs in Kinesis streams #1710) by @heitorlessa
• feat(apigateway): multiple exceptions in exception_handler (feat(apigateway): multiple exceptions in exception_handler #1707) by @sprkem
• docs(apigateway): add all resolvers in testing your code section for accuracy (docs(apigateway): add all resolvers in testing your code section for accuracy #1688) by @bnsouza
• docs(homepage): update default value for POWERTOOLS_DEV (docs(homepage): update default value for POWERTOOLS_DEV #1695) by @dreamorosi

🐛 Bug and hot fixes

• fix(parameters): get_secret correctly return SecretBinary value (fix(parameters): get_secret correctly return SecretBinary value #1717) by @mangoes-git
• fix(apigateway): support nested router decorators (fix(apigateway): support nested router decorators #1709) by @kt-hr

🔧 Maintenance

• chore(deps-dev): bump flake8-builtins from 2.0.0 to 2.0.1 (chore(deps-dev): bump flake8-builtins from 2.0.0 to 2.0.1 #1715) by @dependabot
• chore(deps-dev): bump pytest-asyncio from 0.20.1 to 0.20.2 (chore(deps-dev): bump pytest-asyncio from 0.20.1 to 0.20.2 #1723) by @dependabot
• chore(deps-dev): bump mypy-boto3-appconfig from 1.25.0 to 1.26.0.post1 (chore(deps-dev): bump mypy-boto3-appconfig from 1.25.0 to 1.26.0.post1 #1722) by @dependabot
• chore(deps-dev): bump mypy-boto3-ssm from 1.26.0.post1 to 1.26.4 (chore(deps-dev): bump mypy-boto3-ssm from 1.26.0.post1 to 1.26.4 #1721) by @dependabot
• chore(deps-dev): bump mypy-boto3-xray from 1.26.0.post1 to 1.26.9 (chore(deps-dev): bump mypy-boto3-xray from 1.26.0.post1 to 1.26.9 #1720) by @dependabot
• chore(deps-dev): bump mypy-boto3-lambda from 1.25.0 to 1.26.0.post1 (chore(deps-dev): bump mypy-boto3-lambda from 1.25.0 to 1.26.0.post1 #1705) by @dependabot
• chore(deps-dev): bump mypy-boto3-s3 from 1.25.0 to 1.26.0.post1 (chore(deps-dev): bump mypy-boto3-s3 from 1.25.0 to 1.26.0.post1 #1716) by @dependabot
• chore(deps-dev): bump mypy-boto3-appconfigdata from 1.25.0 to 1.26.0.post1 (chore(deps-dev): bump mypy-boto3-appconfigdata from 1.25.0 to 1.26.0.post1 #1704) by @dependabot
• chore(deps-dev): bump mypy-boto3-xray from 1.25.0 to 1.26.0.post1 (chore(deps-dev): bump mypy-boto3-xray from 1.25.0 to 1.26.0.post1 #1703) by @dependabot
• chore(deps-dev): bump mypy-boto3-cloudwatch from 1.25.0 to 1.26.0.post1 (chore(deps-dev): bump mypy-boto3-cloudwatch from 1.25.0 to 1.26.0.post1 #1714) by @dependabot
• chore(logger): overload inject_lambda_context with generics (chore(logger): overload inject_lambda_context with generics #1583) by @ascopes
• chore(deps-dev): bump types-requests from 2.28.11.3 to 2.28.11.4 (chore(deps-dev): bump types-requests from 2.28.11.3 to 2.28.11.4 #1701) by @dependabot
• chore(deps-dev): bump mypy-boto3-logs from 1.25.0 to 1.26.3 (chore(deps-dev): bump mypy-boto3-logs from 1.25.0 to 1.26.3 #1702) by @dependabot
• chore(deps-dev): bump pytest-xdist from 2.5.0 to 3.0.2 (chore(deps-dev): bump pytest-xdist from 2.5.0 to 3.0.2 #1655) by @dependabot
• chore(deps-dev): bump mkdocs-material from 8.5.7 to 8.5.9 (chore(deps-dev): bump mkdocs-material from 8.5.7 to 8.5.9 #1697) by @dependabot
• chore(deps-dev): bump flake8-comprehensions from 3.10.0 to 3.10.1 (chore(deps-dev): bump flake8-comprehensions from 3.10.0 to 3.10.1 #1699) by @dependabot
• chore(deps-dev): bump types-requests from 2.28.11.2 to 2.28.11.3 (chore(deps-dev): bump types-requests from 2.28.11.2 to 2.28.11.3 #1698) by @dependabot
• chore(deps-dev): bump pytest-benchmark from 3.4.1 to 4.0.0 (chore(deps-dev): bump pytest-benchmark from 3.4.1 to 4.0.0 #1659) by @dependabot
• chore(deps-dev): bump mypy-boto3-secretsmanager from 1.25.0 to 1.26.0.post1 (chore(deps-dev): bump mypy-boto3-secretsmanager from 1.25.0 to 1.26.0.post1 #1691) by @dependabot
• chore(deps): bump dependabot/fetch-metadata from 1.3.4 to 1.3.5 (chore(deps): bump dependabot/fetch-metadata from 1.3.4 to 1.3.5 #1689) by @dependabot
• chore(deps-dev): bump flake8-bugbear from 22.10.25 to 22.10.27 (chore(deps-dev): bump flake8-bugbear from 22.10.25 to 22.10.27 #1665) by @dependabot
• chore(deps-dev): bump mypy-boto3-ssm from 1.25.0 to 1.26.0.post1 (chore(deps-dev): bump mypy-boto3-ssm from 1.25.0 to 1.26.0.post1 #1690) by @dependabot

This release was made possible by the following contributors:

@ascopes, @bnsouza, @dependabot, @dependabot[bot], @dreamorosi, @heitorlessa, @kt-hr, @mangoes-git, @sprkem and Release bot

---

This discussion was created from the release v2.3.0.