stitchfix · skrawcz · Jan 31, 2023 · Jan 7, 2023 · Jan 10, 2023 · Jan 10, 2023
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
@@ -0,0 +1,24 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  fail_on_warning: false
+  configuration: docs/conf.py
+
+# Optionally set the version of Python and requirements required to build your docs
+python:
+  install:
+    - requirements: requirements-docs.txt
+    - method: pip
+      path: .
diff --git a/docs/_static/image.png b/docs/_static/image.png
diff --git a/docs/best-practices/code-organization.rst b/docs/best-practices/code-organization.rst
@@ -0,0 +1,3 @@
+=================
+Code Organization
+=================
diff --git a/docs/best-practices/common-indices.rst b/docs/best-practices/common-indices.rst
@@ -0,0 +1,3 @@
+==============
+Common Indices
+==============
diff --git a/docs/best-practices/function-modifiers.rst b/docs/best-practices/function-modifiers.rst
@@ -0,0 +1,3 @@
+==================
+Function Modifiers
+==================
diff --git a/docs/best-practices/function-naming.rst b/docs/best-practices/function-naming.rst
@@ -0,0 +1,3 @@
+===============
+Function Naming
+===============
diff --git a/docs/best-practices/index.rst b/docs/best-practices/index.rst
@@ -0,0 +1,15 @@
+==============
+Best Practices
+==============
+
+.. toctree::
+   :hidden:
+
+   using-within-your-etl-system
+   migrating-to-hamilton
+   function-naming
+   code-organization
+   function-modifiers
+   common-indices
+   loading-data
+   output-immutability
diff --git a/docs/best-practices/loading-data.rst b/docs/best-practices/loading-data.rst
@@ -0,0 +1,3 @@
+============
+Loading Data
+============
diff --git a/docs/best-practices/migrating-to-hamilton.rst b/docs/best-practices/migrating-to-hamilton.rst
@@ -0,0 +1,3 @@
+=====================
+Migrating to Hamilton
+=====================
diff --git a/docs/best-practices/output-immutability.rst b/docs/best-practices/output-immutability.rst
@@ -0,0 +1,3 @@
+===================
+Output Immutability
+===================
diff --git a/docs/best-practices/using-within-your-etl-system.rst b/docs/best-practices/using-within-your-etl-system.rst
@@ -0,0 +1,3 @@
+============================
+Using within your ETL System
+============================
diff --git a/docs/conf.py b/docs/conf.py
@@ -0,0 +1,9 @@
+project = "Hamilton"
+
+html_theme = "furo"
+html_title = "Hamilton"
+html_theme_options = {
+    "source_repository": "https://github.com/stitchfix/hamilton",
+    "source_branch": "main",
+    "source_directory": "docs/",
+}
diff --git a/docs/contributing.rst b/docs/contributing.rst
@@ -0,0 +1,3 @@
+============
+Contributing
+============
diff --git a/docs/extensions.rst b/docs/extensions.rst
@@ -0,0 +1,3 @@
+==========
+Extensions
+==========
diff --git a/docs/hamilton-community.rst b/docs/hamilton-community.rst
@@ -0,0 +1,3 @@
+==================
+Hamilton Community
+==================
diff --git a/docs/index.rst b/docs/index.rst
@@ -0,0 +1,162 @@
+===============
+Hamilton
+===============
+
+.. toctree::
+   :maxdepth: 6
+   :hidden:
+
+   less-than-15-minutes-to-mastery/index
+
+.. toctree::
+   :maxdepth: 6
+   :hidden:
+
+   overview-of-concepts/index
+
+.. toctree::
+   :maxdepth: 6
+   :hidden:
+
+   tutorials/index
+
+.. toctree::
+   :maxdepth: 6
+   :hidden:
+
+   best-practices/index
+
+.. toctree::
+   :maxdepth: 6
+   :hidden:
+
+   extensions
+   talks-or-podcasts-or-blogs-or-papers
+   hamilton-community
+   license
+   contributing
+
+.. toctree::
+   :maxdepth: 6
+   :caption: REFERENCE
+   :hidden:
+
+   reference/api-extensions/index
+   reference/api-reference/index
+
+.. toctree::
+   :maxdepth: 6
+   :caption: UNORGANIZED-DOCS
+   :hidden:
+
+   unorganized-docs/index
+
+The open source framework `Hamilton <https://github.com/stitchfix/hamilton>`_, originally built to manage and run Stitch
+Fix's data pipelines.
+
+.. _getting started:
+
+Getting Started
+---------------
+
+If you want to jump in head first, we have a simple tutorial for getting started!  To ask questions, please join our
+`slack community <https://join.slack.com/t/hamilton-opensource/shared\_invite/zt-1bjs72asx-wcUTgH7q7QX1igiQ5bbdcg>`_!
+
+:doc:`less-than-15-minutes-to-mastery/index`
+
+.. _what is hamilton:
+
+What is Hamilton?
+-----------------
+
+It's a general purpose micro-framework for creating `dataflows <https://en.wikipedia.org/wiki/Dataflow>`_ from python
+functions!
+
+Specifically, Hamilton defines a novel paradigm, that allows you to specify a flow of (delayed) execution, that forms a 
+Directed Acyclic Graph (DAG). It was original built to solve creating wide (1000+) column dataframes at Stitch Fix. Core 
+to the design of Hamilton is a clear mapping of function name to dataflow output. That is, Hamilton forces a certain 
+paradigm with writing functions, and aims for DAG clarity, easy modifications, _with always unit testable and naturally 
+documentable code!
+
+For the backstory on how Hamilton came about, see our
+`blog post <https://multithreaded.stitchfix.com/blog/2021/10/14/functions-dags-hamilton/>`_!
+
+Hamilton's method of defining dataflows presents a new paradigm when it comes to creating, um, dataframes (let's use 
+dataframes as an example, otherwise you can create _ANY_ python object). Rather than thinking about manipulating a 
+central dataframe procedurally, and extracting the data you want, as is normal in some data engineering/data science 
+work, you instead think about the column(s) (a.k.a. outputs) you want to create, and what inputs are required. 
+There is no need for you to think about maintaining how to create this dataframe, meaning you do not need to think 
+about any "glue" code; this is all taken care of by the Hamilton framework. Specifically, Hamilton enables you to run 
+your dataflow with the following steps:
+
+1. Define your dataflow (a.k.a. pipeline), as a set of transforms using Hamilton's paradigm for writing python functions.
+2. Specify the parameter values your dataflow requires
+3. Specify which outputs you want to compute from your dataflow
+
+The rest is delegated to the framework, which handles the computation for you.
+
+Let's illustrate this with some code. If you were asked to write a simple transform (let's use pandas for the sake of 
+argument), you may decide to write something simple like this:
+
+.. code-block:: python
+
+    df['col_c'] = df['col_a'] + df['col_b']
+
+To represent this in a way Hamilton can understand, you write:
+
+.. code-block:: python
+
+    def col_c(col_a: pd.Series, col_b: pd.Series) -> pd.Series:
+        """Creating column c from summing column a and column b."""
+        return col_a + col_b
+
+.. image:: _static/image.png
+    :alt: The above code represented as a diagram
+
+The Hamilton framework takes the above code, forms it into a computational DAG, and executes it for you!
+
+.. _hamilton open source community:
+
+Hamilton Open Source Community
+------------------------------
+
+If you have questions, have ideas, or need help with Hamilton, join us in our
+`slack community <https://join.slack.com/t/hamilton-opensource/shared\_invite/zt-1bjs72asx-wcUTgH7q7QX1igiQ5bbdcg>`_,
+and we'll try to help!
+
+.. _installing hamilton:
+
+Installing Hamilton
+-------------------
+
+Installation should be quick and as simple as:
+
+.. code-block:: sh
+
+    > pip install sf-hamilton
+
+For more information please see :doc:`less-than-15-minutes-to-mastery/installing`.
+
+.. _license:
+
+License
+---------------
+
+Hamilton is released under the `BSD 3-Clause Clear License <https://github.com/stitchfix/hamilton/blob/main/LICENSE>`_.
+If you need to get in touch about something, contact us at algorithms-opensource (at) stitchfix.com.
+
+.. _contributing:
+
+Contributing
+---------------
+
+We take contributions, large and small. We operate via a Code of Conduct and expect anyone contributing to do the same.
+
+.. _user guide:
+
+User Guide
+---------------
+
+Dive a little deeper and start exploring our API reference to get an idea of everything that's possible with the API:
+
+:doc:`reference/api-reference/index`
diff --git a/docs/less-than-15-minutes-to-mastery/digging-in-further.rst b/docs/less-than-15-minutes-to-mastery/digging-in-further.rst
@@ -0,0 +1,3 @@
+==================
+Digging in Further
+==================
diff --git a/docs/less-than-15-minutes-to-mastery/index.rst b/docs/less-than-15-minutes-to-mastery/index.rst
@@ -0,0 +1,11 @@
+======================
+<15 Minutes to Mastery
+======================
+
+.. toctree::
+   :hidden:
+
+   installing
+   writing-your-first-dataflow
+   running-your-dataflow
+   digging-in-further
diff --git a/docs/less-than-15-minutes-to-mastery/installing.rst b/docs/less-than-15-minutes-to-mastery/installing.rst
@@ -0,0 +1,28 @@
+==========
+Installing
+==========
+
+---
+description: Installing hamilton is easy!
+---
+
+## Requirements
+
+Hamilton is a lightweight framework with a variety of extensions/plugins. To get started, you'll need the following:
+
+* [ ] `python >= 3.6`
+* [ ] `pip`
+
+For help with python/pip/managing virtual environments see the [python docs](https://docs.python.org/3/tutorial/venv.html).
+
+## **Installing**
+
+Hamilton is published on [pypi](https://pypi.org/project/sf-hamilton/1.1.1/) under `sf-hamilton`. To install, run:
+
+`pip install sf-hamilton`
+
+To use the DAG visualization functionality, instead install with&#x20;
+
+`pip install sf-hamilton[visualization]`
+
+``Note: for visualization you may additionally need to install `graphviz` externally -- see`` [`https://graphviz.org/download/`](https://graphviz.org/download/) `for instructions on the correct way for your operating system.`
diff --git a/docs/less-than-15-minutes-to-mastery/running-your-dataflow.rst b/docs/less-than-15-minutes-to-mastery/running-your-dataflow.rst
@@ -0,0 +1,3 @@
+=====================
+Running Your Dataflow
+=====================
diff --git a/docs/less-than-15-minutes-to-mastery/writing-your-first-dataflow.rst b/docs/less-than-15-minutes-to-mastery/writing-your-first-dataflow.rst
@@ -0,0 +1,3 @@
+===========================
+Writing Your First Dataflow
+===========================
diff --git a/docs/license.rst b/docs/license.rst
@@ -0,0 +1,3 @@
+=======
+License
+=======
diff --git a/docs/overview-of-concepts/decorators.rst b/docs/overview-of-concepts/decorators.rst
@@ -0,0 +1,3 @@
+==========
+Decorators
+==========
diff --git a/docs/overview-of-concepts/index.rst b/docs/overview-of-concepts/index.rst
@@ -0,0 +1,11 @@
+====================
+Overview of Concepts
+====================
+
+.. toctree::
+   :hidden:
+
+   lexicon
+   writing-transform-functions
+   the-hamilton-driver
+   decorators
diff --git a/docs/overview-of-concepts/lexicon.rst b/docs/overview-of-concepts/lexicon.rst
@@ -0,0 +1,3 @@
+=======
+Lexicon
+=======
diff --git a/docs/overview-of-concepts/the-hamilton-driver.rst b/docs/overview-of-concepts/the-hamilton-driver.rst
@@ -0,0 +1,3 @@
+===================
+The Hamilton Driver
+===================
diff --git a/docs/overview-of-concepts/writing-transform-functions.rst b/docs/overview-of-concepts/writing-transform-functions.rst
@@ -0,0 +1,3 @@
+===========================
+Writing Transform Functions
+===========================
diff --git a/docs/reference/api-extensions/index.rst b/docs/reference/api-extensions/index.rst
@@ -0,0 +1,3 @@
+==============
+API Extensions
+==============
diff --git a/docs/reference/api-reference/index.rst b/docs/reference/api-reference/index.rst
@@ -0,0 +1,3 @@
+=============
+API Reference
+=============
diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -0,0 +1 @@
+mkdocs==1.4.2
diff --git a/docs/talks-or-podcasts-or-blogs-or-papers.rst b/docs/talks-or-podcasts-or-blogs-or-papers.rst
@@ -0,0 +1,3 @@
+========================
+Talks | Podcasts | Blogs
+========================
diff --git a/docs/tutorials/dask.rst b/docs/tutorials/dask.rst
@@ -0,0 +1,3 @@
+====
+Dask
+====
diff --git a/docs/tutorials/data-quality-checks.rst b/docs/tutorials/data-quality-checks.rst
@@ -0,0 +1,3 @@
+===================
+Data Quality Checks
+===================
diff --git a/docs/tutorials/index.rst b/docs/tutorials/index.rst
@@ -0,0 +1,14 @@
+=========
+Tutorials
+=========
+
+.. toctree::
+   :hidden:
+
+   using-hamilton-in-a-notebook
+   integrating-decorators
+   data-quality-checks
+   ray
+   model-training
+   dask
+   spark
diff --git a/docs/tutorials/integrating-decorators.rst b/docs/tutorials/integrating-decorators.rst
@@ -0,0 +1,3 @@
+======================
+Integrating Decorators
+======================
diff --git a/docs/tutorials/model-training.rst b/docs/tutorials/model-training.rst
@@ -0,0 +1,3 @@
+==============
+Model Training
+==============
diff --git a/docs/tutorials/ray.rst b/docs/tutorials/ray.rst
@@ -0,0 +1,3 @@
+======
+⚡ Ray
+======
diff --git a/docs/tutorials/spark.rst b/docs/tutorials/spark.rst
@@ -0,0 +1,3 @@
+=====
+Spark
+=====
-Original file line number
+Diff line change
@@ -0,0 +1,3 @@
+    =======
+    License
+    =======