Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

examples: Jaffle shop example added #799

Merged
merged 3 commits into from
Apr 2, 2024
Merged

examples: Jaffle shop example added #799

merged 3 commits into from
Apr 2, 2024

Conversation

zilto
Copy link
Collaborator

@zilto zilto commented Apr 1, 2024

This repository is a reimplementation of the canonical dbt example jaffle_shop. Here, Hamilton + Ibis provides a Python-centric alternative to dbt (Learn more). It illustrates data transformations for an ecommerce store. Since it is used by many different frameworks, it can ground comparisons between tools.

Changes

  • added examples/jaffle-shop
  • included data files

How I tested this

  • ran run.py locally

Notes

  • it servers as a good reference for tabular data workflows, but Hamilton supports a much wider set of use cases than dbt

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future TODOs are captured in comments
  • Project documentation has been updated if adding/changing functionality.

@zilto zilto requested a review from skrawcz April 1, 2024 22:33
Copy link
Collaborator

@skrawcz skrawcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

  1. But question on location. Should this instead be under Ibis? Versus standalone jaffle-shop?
  2. Where would a link in the main documentation help?
  3. Would a vanilla pandas version be helpful?

@zilto
Copy link
Collaborator Author

zilto commented Apr 2, 2024

  1. But question on location. Should this instead be under Ibis? Versus standalone jaffle-shop?
  2. Where would a link in the main documentation help?
  3. Would a vanilla pandas version be helpful?
  1. top-level jaffle-shop is a bit easier to find for those that know the name, but I agree to move it to Ibis given it's so tightly coupled.
  2. I would include it in an upcoming docs/code-comparisons/dbt page. Hamilton and dbt might not be direct competitor, but it would give an idea to new users how concepts from translate to the other
  3. it would be more work with little payoff IMHO given the goal is to showcase the "declarative orchestration" rather than "the data transforms". The code seems intelligible even if you don't know Ibis.

Other thoughts:
It would be cool to have a canonical Hamilton project (e.g., Ibis uses penguins dataset, dbt jaffle shop). It should serve the "Getting started" step-by-step tutorial, and be extensible to be able to showcase Hamilton features (add data validation, change execution engine, etc.). Will open an issue to spec this

@zilto zilto merged commit b66e657 into main Apr 2, 2024
23 checks passed
@zilto zilto deleted the examples/jaffle-shop branch April 2, 2024 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants