@pipe decorator #398

elijahbenizzy · 2023-09-28T22:58:30Z

See #65, #372

Changes

Add pipe decorator

How I tested this

Lots of testing, end-to-end included

Notes

Checklist

PR has an informative and human-readable title (this will be pulled into the release notes)
Changes are limited to a single goal (no scope creep)
Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
Any change in functionality is tested
New functions are documented (with a description, list of inputs, and expected output)
Placeholder code is flagged / future TODOs are captured in comments
Project documentation has been updated if adding/changing functionality.

docs/reference/decorators/with_columns.rst

hamilton/function_modifiers/macros.py

tests/resources/pipe.py

skrawcz · 2023-10-18T20:43:46Z

in person conversation notes:

apply --> step
stick with named
get example added

skrawcz

Looks good. Commits should have more design decision whys for your future self to refer back to.

#372 `pipe` allows for function chaining/redefinition. It has a few caveats (see the documentation + list below), but they're all placeholders -- we will have the chance to improve it over time, as more people use it. - uses a series of `step(fn, *args, **kwargs)` to apply functions in a row - the first parameter is chained through to the next ones - we only support decorating hamilton-friendly functions (kwargable/typed) for now - we have an ability to make it a single node but don't do that yet We also place in/comment out the `@flow` decorator, which is equivalent to `pipe`, but doesn't pass the result into the next function. Thus it can be used to build a procedural subdag as a component of a declarative DAG.

elijahbenizzy force-pushed the pipe branch 3 times, most recently from 517b145 to 0af987d Compare October 16, 2023 23:41

elijahbenizzy temporarily deployed to github-pages October 16, 2023 23:41 — with GitHub Actions Inactive

elijahbenizzy force-pushed the pipe branch from 0af987d to 77fac66 Compare October 17, 2023 00:24

elijahbenizzy changed the title ~~WIP~~ @pipe decorator Oct 17, 2023

elijahbenizzy force-pushed the pipe branch 12 times, most recently from b556623 to 03d2803 Compare October 17, 2023 04:33

elijahbenizzy requested a review from skrawcz October 17, 2023 04:36