Jupyter Notebook for Data Loaders Example (Data loaders #1122)

[eoagyen]

This PR adds a Jupyter Notebook for the data_loaders example to provide a less intimidating entry point for data scientists who prefer using notebooks over Python scripts. The notebook mirrors the functionality of the existing run.py file, enabling users to experiment and try things out interactively.

Changes

A Jupyter notebook was added for the data_loaders example. The notebook mirrors the functionality of run.py.

How I tested this

Tested the notebook by running all cells and verifying that the outputs match those from run.py. Ensured that all functions and processes work as expected within the notebook environment.

Notes

• This is part of an ongoing effort to provide Jupyter notebooks for all examples to make them more accessible for data scientists.
• No major changes were made to the underlying code; the notebook serves as an alternative to run.py.

Checklist

• PR has an informative and human-readable title (this will be pulled into the release notes)
• Changes are limited to a single goal (no scope creep)
• Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future TODOs are captured in comments
• Project documentation has been updated if adding/changing functionality.

[skrawcz]

@eoagyen thanks!

Some feedback -- would you be able to use the %%cell_to_module magic for the notebook?
E.g. for the code you show, you'll be able to add that to the top, see https://github.com/DAGWorks-Inc/hamilton/tree/main/examples/jupyter_notebook_magic for how it works.

So shouldn't be a big change -- just removing the import cell, using %%cell_to_module and adding the right imports.

e.g. do something like this at the top of the respective cells that define Hamilton functions

%%cell_to_module load_data_csv --display
# Loading csv data
import pandas as pd
from hamilton.function_modifiers import load_from, does, extract_columns, parameterize, source, value

You can then remove the import of load_data_csv because that cell will define it.

[skrawcz]

Can you use the newer builder pattern please?
e.g.

from hamilton import base
import hamilton.driver

driver = (hamilton.driver.Builder()
      .with_modules(load_data_duckdb, prep_data)
      .with_adapters(base.PandasDataFrameResult()
      .build()
)

then for the visualization pass in inputs with the dictionary:

duckdb_execution_graph = driver.visualize_execution(VARS, inputs={"db_path": "./test_data/database.duckdb"})

[eoagyen]

Thank you @skrawcz, I have implemented the changes suggested.

[skrawcz]

Thanks @eoagyen !