Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add decorator factory to create your own middleware #17

Merged
merged 37 commits into from
Apr 20, 2020

Conversation

heitorlessa
Copy link
Contributor

@heitorlessa heitorlessa commented Apr 10, 2020

Issue #, if available: #20

Description of changes:

This is required for Serverless Airline project to move from the modified version of powertools, and create own middlewares to improve reusability as much as possible.

Instead of creating reusable methods, we're at the stage that makes more sense to give flexibility to create own middlewares as #9, so customers can create one middleware with many custom processing before, after and exception handling instead.

This gives flexibility to control before, after, and exception handling logic in the way you'd like. It also lay the foundations to allow us to create a base Trace provider so you can use any Tracer that follow our protocol.

  • Create decorator factory
  • Tests
  • Use existing Tracer class instance to trace custom middlewares
  • Update existing middlewares
  • Update docs
  • Utility to optionally trace middleware
  • Optional: Automate API docs

One step closer from GA 🎉 - We'll tackle log level next!

User Experience

Create a middleware no params

from aws_lambda_powertools.middleware_factory import lambda_handler_decorator

@lambda_handler_decorator
def log_response(handler, event, context):
    any_code_to_execute_before_lambda_handler()                
    response = handler(event, context)
    any_code_to_execute_after_lambda_handler()
    print(f"Lambda handler response: {response}")
    return response

@log_response
def lambda_handler(event, context):
    return True

Create a middleware with params

from aws_lambda_powertools.middleware_factory import lambda_handler_decorator

@lambda_handler_decorator
def obfuscate_sensitive_data(handler, event, context, fields=None):
    # Obfuscate email before calling Lambda handler
    if fields:
        for field in fields:
            field = event.get(field, "")
            event[field] = obfuscate_pii(field)

    return handler(event, context)
    

@obfuscate_sensitive_data(fields=["email"])
def lambda_handler(event, context):
    return True

Trace execution of custom middleware - Creates sub-segments in X-Ray

```python
from aws_lambda_powertools.utils import lambda_handler_decorator

@lambda_handler_decorator(trace_execution=True)
def my_custom_middleware(handler, event, context):
    ...

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@heitorlessa
Copy link
Contributor Author

heitorlessa commented Apr 12, 2020

Local perf tests to rule out major performance impact on multiple callables - Decorator only prints before handler and after.

While Lambda will have less CPU it'll continue to be less than a microsecond, so it worth using this factory to refactor codebase to make it more contributor friendly (dev time is more expensive)

No decorator

50000000 loops, best of 5: 6.88 nsec per loop

Simpler decorator

50000000 loops, best of 5: 7.18 nsec per loop

Decorator factory

50000000 loops, best of 5: 7.25 nsec per loop

Simpler Stacked decorator

50000000 loops, best of 5: 7.05 nsec per loop

Factory Stacked decorator

50000000 loops, best of 5: 7.05 nsec per loop


python -m timeit -s 'import handler; handler.lambda_handler({}, {})'

def call_handler_simpler(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print("SIMPLER - BEGIN")
        response = func(*args, **kwargs)
        print("SIMPLER - END")
        return response
    return wrapper

@lambda_handler_decorator
def call_handler_factory(handler, event, context):
    print("[CALL HANDLER] BEGIN")
    response = handler(event, context)
    print("[CALL HANDLER] END")
    return response

def lambda_handler(event, context):
    a = {
        "statusCode": 200,
        "body": "Hello world"
    }
    print("-------------LAMBDA FN CALLED!-------------")
    return a

@heitorlessa heitorlessa changed the title [WIP] Decorator factory Feat: Create your own middleware Decorator factory Feat: Create your own middleware Apr 18, 2020
python/README.md Show resolved Hide resolved
python/README.md Outdated Show resolved Hide resolved
python/README.md Show resolved Hide resolved
python/aws_lambda_powertools/tracing/tracer.py Outdated Show resolved Hide resolved
python/aws_lambda_powertools/middleware_factory/factory.py Outdated Show resolved Hide resolved
python/aws_lambda_powertools/middleware_factory/factory.py Outdated Show resolved Hide resolved
@heitorlessa heitorlessa merged commit 6ce07e9 into develop Apr 20, 2020
@heitorlessa heitorlessa deleted the feat/dec_factory branch April 20, 2020 15:56
heitorlessa added a commit that referenced this pull request Apr 21, 2020
* docs: add pypi badge

* fix: add missing single_metric example; test var name

* chore: pypi monthly download badge

* chore: fix github badge typo

* feat: add docs to CI

* fix: CI attempt 2

* fix: CI attempt 3

* fix: CI attempt 3

* fix: CI attempt 4

* chore: clean up CI workflows

* Decorator factory Feat: Create your own middleware (#17)

* feat(utils): add decorator factory

* improv: use partial to reduce complexity

* improv: add error handling

* chore: type hint

* docs: include pypi downloads badge

* feat: opt in to trace each middleware that runs

* improv: add initial util tests

* improv: test explicit and implicit trace_execution

* improv: test decorator with params

* chore: linting

* docs: include utilities

* improv: correct tests, dec_factory only for func

* improv: make util name more explicit

* improv: doc trace_execution, fix casting

* docs: add limitations, improve syntax

* docs: use new docs syntax

* fix: remove middleware decorator from libs

* feat: build docs in CI

* chore: linting

* fix: CI python-version type

* chore: remove docs CI

* chore: kick CI

* chore: include build badge master branch

* chore: refactor naming

* fix: rearrange tracing tests

* improv(tracer): toggle default auto patching

* feat(tracer): retrieve registered class instance

* fix(Makefile):  make cov target more explicit

* improv(Register): support multiple classes reg.

* improv(Register): inject class methods correctly

* docs: add how to reutilize Tracer

* improv(tracer): test auto patch method

* improv: address nicolas feedback

* improv: update example to reflect middleware feat

* fix: metric dimension in root blob

* chore: version bump

Co-authored-by: heitorlessa <lessa@amazon.co.uk>

Co-authored-by: heitorlessa <lessa@amazon.co.uk>
heitorlessa added a commit that referenced this pull request May 16, 2020
* develop: (21 commits)
  bugfix: #32 Runtime Error for nested sync fns
  chore: renamed history to changelog dependabot
  bugfix: resolves #31 aiohttp lazy import
  chore: grammar issues
  improv: add project tenets
  Improv tracer - async support, patch, test coverage and X-Ray escape hatch (#29)
  Bugfix: "per second" metric units (#27)
  fix: #24 correct example test and docs
  chore: bump example to use 0.8.0 features
  Adopt logging best practices (#23)
  Decorator factory Feat: Create your own middleware (#17)
  chore: clean up CI workflows
  fix: CI attempt 4
  fix: CI attempt 3
  fix: CI attempt 3
  fix: CI attempt 2
  feat: add docs to CI
  chore: fix github badge typo
  chore: pypi monthly download badge
  fix: add missing single_metric example; test var name
  ...
heitorlessa added a commit that referenced this pull request May 29, 2020
* [Sync Master] 0.7.0 release (#22)

* docs: add pypi badge

* fix: add missing single_metric example; test var name

* chore: pypi monthly download badge

* chore: fix github badge typo

* feat: add docs to CI

* fix: CI attempt 2

* fix: CI attempt 3

* fix: CI attempt 3

* fix: CI attempt 4

* chore: clean up CI workflows

* Decorator factory Feat: Create your own middleware (#17)

* feat(utils): add decorator factory

* improv: use partial to reduce complexity

* improv: add error handling

* chore: type hint

* docs: include pypi downloads badge

* feat: opt in to trace each middleware that runs

* improv: add initial util tests

* improv: test explicit and implicit trace_execution

* improv: test decorator with params

* chore: linting

* docs: include utilities

* improv: correct tests, dec_factory only for func

* improv: make util name more explicit

* improv: doc trace_execution, fix casting

* docs: add limitations, improve syntax

* docs: use new docs syntax

* fix: remove middleware decorator from libs

* feat: build docs in CI

* chore: linting

* fix: CI python-version type

* chore: remove docs CI

* chore: kick CI

* chore: include build badge master branch

* chore: refactor naming

* fix: rearrange tracing tests

* improv(tracer): toggle default auto patching

* feat(tracer): retrieve registered class instance

* fix(Makefile):  make cov target more explicit

* improv(Register): support multiple classes reg.

* improv(Register): inject class methods correctly

* docs: add how to reutilize Tracer

* improv(tracer): test auto patch method

* improv: address nicolas feedback

* improv: update example to reflect middleware feat

* fix: metric dimension in root blob

* chore: version bump

Co-authored-by: heitorlessa <lessa@amazon.co.uk>

Co-authored-by: heitorlessa <lessa@amazon.co.uk>

* feat: add algolia search for docs and api ref (#39) (#40)

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* fix: revert makefile build-docs-api

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* fix: metric_set reuse #43

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* fix: clear metrics

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* fix: update serialize_metrics helper function to use MetricManager instead of Metrics

* fix: clear metrics after lambda invocation #43

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* improv: document metrics tests, remove redundants

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

#43

* chore: linting

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* docs: add section to flush metrics manually

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* docs: include EMF Json object

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* chore: bump version 0.9.4

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

Co-authored-by: Tom McCarthy <thommcc@amazon.com>
@heitorlessa heitorlessa changed the title Decorator factory Feat: Create your own middleware feat: add decorator factory to create your own middleware Jun 3, 2020
@heitorlessa heitorlessa added feature New feature or functionality and removed enhancement labels Jun 3, 2020
heitorlessa referenced this pull request in heitorlessa/aws-lambda-powertools-python Jun 17, 2020
* feat(utils): add decorator factory

* improv: use partial to reduce complexity

* improv: add error handling

* chore: type hint

* docs: include pypi downloads badge

* feat: opt in to trace each middleware that runs

* improv: add initial util tests

* improv: test explicit and implicit trace_execution

* improv: test decorator with params

* chore: linting

* docs: include utilities

* improv: correct tests, dec_factory only for func

* improv: make util name more explicit

* improv: doc trace_execution, fix casting

* docs: add limitations, improve syntax

* docs: use new docs syntax

* fix: remove middleware decorator from libs

* feat: build docs in CI

* chore: linting

* fix: CI python-version type

* chore: remove docs CI

* chore: kick CI

* chore: include build badge master branch

* chore: refactor naming

* fix: rearrange tracing tests

* improv(tracer): toggle default auto patching

* feat(tracer): retrieve registered class instance

* fix(Makefile):  make cov target more explicit

* improv(Register): support multiple classes reg.

* improv(Register): inject class methods correctly

* docs: add how to reutilize Tracer

* improv(tracer): test auto patch method

* improv: address nicolas feedback

* improv: update example to reflect middleware feat

* fix: metric dimension in root blob

* chore: version bump

Co-authored-by: heitorlessa <lessa@amazon.co.uk>
heitorlessa referenced this pull request in heitorlessa/aws-lambda-powertools-python Jun 17, 2020
* [Sync Master] 0.7.0 release (#22)

* docs: add pypi badge

* fix: add missing single_metric example; test var name

* chore: pypi monthly download badge

* chore: fix github badge typo

* feat: add docs to CI

* fix: CI attempt 2

* fix: CI attempt 3

* fix: CI attempt 3

* fix: CI attempt 4

* chore: clean up CI workflows

* Decorator factory Feat: Create your own middleware (#17)

* feat(utils): add decorator factory

* improv: use partial to reduce complexity

* improv: add error handling

* chore: type hint

* docs: include pypi downloads badge

* feat: opt in to trace each middleware that runs

* improv: add initial util tests

* improv: test explicit and implicit trace_execution

* improv: test decorator with params

* chore: linting

* docs: include utilities

* improv: correct tests, dec_factory only for func

* improv: make util name more explicit

* improv: doc trace_execution, fix casting

* docs: add limitations, improve syntax

* docs: use new docs syntax

* fix: remove middleware decorator from libs

* feat: build docs in CI

* chore: linting

* fix: CI python-version type

* chore: remove docs CI

* chore: kick CI

* chore: include build badge master branch

* chore: refactor naming

* fix: rearrange tracing tests

* improv(tracer): toggle default auto patching

* feat(tracer): retrieve registered class instance

* fix(Makefile):  make cov target more explicit

* improv(Register): support multiple classes reg.

* improv(Register): inject class methods correctly

* docs: add how to reutilize Tracer

* improv(tracer): test auto patch method

* improv: address nicolas feedback

* improv: update example to reflect middleware feat

* fix: metric dimension in root blob

* chore: version bump

Co-authored-by: heitorlessa <lessa@amazon.co.uk>

Co-authored-by: heitorlessa <lessa@amazon.co.uk>

* feat: add algolia search for docs and api ref (#39) (#40)

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* fix: revert makefile build-docs-api

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* fix: metric_set reuse #43

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* fix: clear metrics

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* fix: update serialize_metrics helper function to use MetricManager instead of Metrics

* fix: clear metrics after lambda invocation #43

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* improv: document metrics tests, remove redundants

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

#43

* chore: linting

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* docs: add section to flush metrics manually

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* docs: include EMF Json object

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

* chore: bump version 0.9.4

Signed-off-by: heitorlessa <lessa@amazon.co.uk>

Co-authored-by: Tom McCarthy <thommcc@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants