Feature: inline data saver & loaders

[skrawcz]

This change enables one to have inline loaders / savers and to expose metadata for them.

import pandas as pd
from sklearn import datasets

from hamilton.function_modifiers import dataloader, datasaver
from hamilton.io import utils as io_utils


@dataloader()
def raw_data() -> tuple[pd.DataFrame, dict]:
    data = datasets.load_digits()
    df = pd.DataFrame(data.data, columns=[f"feature_{i}" for i in range(data.data.shape[1])])
    metadata = io_utils.get_dataframe_metadata(df)
    return df, metadata


def transformed_data(raw_data: pd.DataFrame) -> pd.DataFrame:
    return raw_data


@datasaver()
def saved_data(transformed_data: pd.DataFrame, filepath: str) -> dict:
    transformed_data.to_csv(filepath)
    metadata = io_utils.get_file_and_dataframe_metadata(filepath, transformed_data)
    return metadata

This results in the following being captured in the tracker / UI

Changes

• adds decorators & docs
• tests
• example

How I tested this

• locally
• unit tests

Notes

Checklist

• PR has an informative and human-readable title (this will be pulled into the release notes)
• Changes are limited to a single goal (no scope creep)
• Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future TODOs are captured in comments
• Project documentation has been updated if adding/changing functionality.

[skrawcz]

Okay let's instead go for the following if it's simpler to implement:

from hamilton.function_modifiers import loader, saver
from hamilton.io import utils as io_utils

@loader # injects node to pull out result
def foo() -> tuple[pd.DataFrame, dict]:
   ...
   metadata = io_utils....(file, df)
   return DF, metadata

@saver # all it does is add the right tags
def write_foo(...) -> dict:
   ...
   metadata = io_utils....(file)
   return metadata

[elijahbenizzy]

Okay let's instead go for the following if it's simpler to implement:

from hamilton.function_modifiers import loader, saver

from hamilton.io import utils as io_utils



@loader # injects node to pull out result

def foo() -> tuple[pd.DataFrame, dict]:

   ...

   metadata = io_utils....(file, df)

   return DF, metadata



@saver # all it does is add the right tags

def write_foo(...) -> dict:

   ...

   metadata = io_utils....(file)

   return metadata

I also find it clearer :)

[elijahbenizzy]

Didn't look for polish, otherwise a few comments, looks good