Skip to content

Commit

Permalink
Update docs for with_columns
Browse files Browse the repository at this point in the history
Add docs for new with_columns implementations.
  • Loading branch information
jernejfrank authored and elijahbenizzy committed Nov 27, 2024
1 parent 2ec6d17 commit a483a07
Show file tree
Hide file tree
Showing 4 changed files with 27 additions and 7 deletions.
24 changes: 21 additions & 3 deletions docs/reference/decorators/with_columns.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,35 @@
with_columns
=======================

Pandas
--------------
We support the `with_columns` operation that appends the results as new columns to the original dataframe for several libraries:

We have a ``with_columns`` option to run operations on columns of a Pandas dataframe and append the results as new columns.
Pandas
-----------------------

**Reference Documentation**

.. autoclass:: hamilton.plugins.h_pandas.with_columns
:special-members: __init__


Polar (Eager)
-----------------------

**Reference Documentation**

.. autoclass:: hamilton.plugins.h_polars.with_columns
:special-members: __init__


Polars (Lazy)
-----------------------

**Reference Documentation**

.. autoclass:: hamilton.plugins.h_polars_lazyframe.with_columns
:special-members: __init__


PySpark
--------------

Expand Down
2 changes: 1 addition & 1 deletion examples/pandas/with_columns/README
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

We show the ability to use the familiar `with_columns` from either `pyspark` or `polars` on a Pandas dataframe.

To see the example look at the notebook.
To see the example look at the [notebook](notebook.ipynb).

![image info](./dag.png)
4 changes: 2 additions & 2 deletions examples/polars/with_columns/README
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Using with_columns with Polars

We show the ability to use the familiar `with_columns` from `polars`. Supported for both `pl.DataFrame` and `pl.LazyFrame`.
We show the ability to use the familiar `with_columns` from `polars`. Supported for both: `pl.DataFrame` and `pl.LazyFrame`.

To see the example look at the notebook.
To see the example look at the [notebook](notebook.ipynb).

![image info](./DAG_DataFrame.png)
![image info](./DAG_lazy.png)
4 changes: 3 additions & 1 deletion hamilton/function_modifiers/README
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# with_columns_base

Documenting the current design flow for the `with_columns` decorator. It belongs to the `NodeInjector` lifecycle.
Documenting the current design flow for the `with_columns` decorator.

For now, it belongs to the `NodeInjector` lifecycle since it still runs the decorated function as a node but injects the dataframe with columns appended columns as one of the parameters.

The `with_columns` consists of three parts that are represented in the corresponding three abstract methods in `with_columns_base`:

Expand Down

0 comments on commit a483a07

Please sign in to comment.