Skip to content

Commit

Permalink
Modifies hardcoded links in docs to use reST syntax (stitchfix#309)
Browse files Browse the repository at this point in the history
This is a user contribution from 37323048+CharityKithaka@users.noreply.github.com
where the clean up is around the hard coded external links that were not following
the reST syntax properly.

---------

Co-authored-by: Charity Kithaka <37323048+CharityKithaka@users.noreply.github.com>
  • Loading branch information
CharityKithaka authored Feb 15, 2023
1 parent 98a4e7a commit f0ffb4c
Show file tree
Hide file tree
Showing 10 changed files with 34 additions and 37 deletions.
16 changes: 8 additions & 8 deletions docs/best-practices/using-within-your-etl-system.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,21 @@ Compatibility Matrix

* - Framework / Scheduler
- Compatibility
* - `Airflow <http://airflow.org/>`_
* - `Airflow <http://airflow.org>`_
- ✅
* - `Dagster <https://dagster.io/>`_
* - `Dagster <https://dagster.io>`_
- ✅
* - `Prefect <https://prefect.io/>`_
* - `Prefect <https://prefect.io>`_
- ✅
* - `Kubeflow Pipelines <https://www.kubeflow.org/docs/components/pipelines/>`_
* - `Kubeflow Pipelines <https://www.kubeflow.org/docs/components/pipelines>`_
- ✅
* - `CRON <https://en.wikipedia.org/wiki/Cron>`_
- ✅
* - `dbt <https://getdbt.com/>`_
* - `dbt <https://getdbt.com>`_
- ❔ (dbt did not run python, but now it does so it should)
* - `kubernetes <https://kubernetes.io/>`_
- ✅ but you need to setup kubernetes to run an image that can run python code - e.g. see `https://medium.com/avmconsulting-blog/running-a-python-application-on-kubernetes-aws-56609e7cd88c <https://medium.com/avmconsulting-blog/running-a-python-application-on-kubernetes-aws-56609e7cd88c>`_
* - `docker <https://www.docker.com/>`_
* - `kubernetes <https://kubernetes.io>`_
- ✅ but you need to setup kubernetes to run an image that can run python code - e.g. see `Running a python application on kubernetes <https://medium.com/avmconsulting-blog/running-a-python-application-on-kubernetes-aws-56609e7cd88c>`_
* - `docker <https://www.docker.com>`_
- ✅ but you need to setup a docker image that can execute python code.
* - ... in general if it runs python 3.6+ ...
- ✅
Expand Down
15 changes: 7 additions & 8 deletions docs/extensions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,9 @@ endeavor to ensure backwards compatible changes when they can be accommodated.
Ray - Experimental!
===================

`Ray <https://ray.io/>`_ is a system to scale python workloads. Hamilton makes it very easy for you to use Ray.
`Ray <https://ray.io>`_ is a system to scale python workloads. Hamilton makes it very easy for you to use Ray.

See `https://github.com/stitchfix/hamilton/tree/main/examples/ray <https://github.com/stitchfix/hamilton/tree/main/examples/ray>`_
See `Scaling Hamilton on Ray <https://github.com/stitchfix/hamilton/tree/main/examples/ry>`_
for an example of using Ray.

Single Machine:
Expand All @@ -79,19 +79,18 @@ in terms of the amount of data it can process.
Dask - Experimental!
====================

`Ray <https://ray.io/>`_ is a system to scale python workloads. Hamilton makes it very easy for you to use Ray.
`Ray <https://ray.io>`_ is a system to scale python workloads. Hamilton makes it very easy for you to use Ray.

See `https://github.com/stitchfix/hamilton/tree/main/examples/dask <https://github.com/stitchfix/hamilton/tree/main/examples/dask>`_
See `Scaling Hamilton on Dask <https://github.com/stitchfix/hamilton/tree/main/examples/dask>`_
for an example of using Dask to scale Hamilton computation.

Single Machine:
***************

`Dask <https://dask.org/>`_ is a very easy way to enable multi-processing on a single machine. This enables you to
`Dask <https://dask.org>`_ is a very easy way to enable multi-processing on a single machine. This enables you to
easily make use of multiple CPU cores.

What this doesn't help with is data scale, as you're still limited to what fits in memory on your machine.

What this doesn't help with is data scale, as you're still limited to what fits in memory on your machine
Distributed Computation:
************************

Expand All @@ -105,7 +104,7 @@ Koalas on Spark, a.k.a. Pandas API on Spark - Experimental!
was the project code name to implement the `Pandas API on top of Spark <https://spark.apache.org/docs/latest/api/python/user\_guide/pandas\_on\_spark/index.html>`_.
Hamilton makes it very easy for you to use Koalas on Spark.

See `https://github.com/stitchfix/hamilton/tree/main/examples/spark <https://github.com/stitchfix/hamilton/tree/main/examples/spark>`_
See `Scaling Hamilton on Koalas <https://github.com/stitchfix/hamilton/tree/main/examples/spark>`_
for an example of using Koalas on Spark to scale Hamilton computation.

Single Machine:
Expand Down
5 changes: 2 additions & 3 deletions docs/hamilton-community.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ If you want to bounce ideas, ask questions, or just see what the small community

We have a bunch of channels on Slack that are designed to help you get the most out of Hamilton!

Slack space name: `https://hamilton-opensource.slack.com/ <https://hamilton-opensource.slack.com/>`_

Slack invite link: `https://join.slack.com/t/hamilton-opensource/shared\_invite/zt-1bjs72asx-wcUTgH7q7QX1igiQ5bbdcg <https://join.slack.com/t/hamilton-opensource/shared\_invite/zt-1bjs72asx-wcUTgH7q7QX1igiQ5bbdcg>`_
Slack space name: `hamilton-opensource <https://hamilton-opensource.slack.com>`_

Slack invite link: `click me to join <https://join.slack.com/t/hamilton-opensource/shared\_invite/zt-1bjs72asx-wcUTgH7q7QX1igiQ5bbdcg>`_
See you there!
4 changes: 2 additions & 2 deletions docs/less-than-15-minutes-to-mastery/installing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Hamilton is a lightweight framework with a variety of extensions/plugins. To get

* ``pip``

For help with python/pip/managing virtual environments see the `python docs <https://docs.python.org/3/tutorial/venv.html>`_.
For help with python/pip/managing virtual environments see the `python docs <https://docs.python.org/3/tutorial/venv.html/>`_.

Installing
----------
Expand All @@ -27,5 +27,5 @@ To use the DAG visualization functionality, instead install with
``pip install sf-hamilton[visualization]``

*Note: for visualization you may additionally need to install graphviz externally -- see*
`https://graphviz.org/download/ <https://graphviz.org/download/>`_ *for instructions on the correct way for your
`graphviz <https://graphviz.org/download/>`_ *for instructions on the correct way for your
operating system.*
2 changes: 1 addition & 1 deletion docs/overview-of-concepts/the-hamilton-driver.rst
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ Visualizing Execution
#####################
Hamilton enables you to quickly and easily visualize your entire DAG, as well as the specific execution path to compute
an output. Underneath we default to use [graphviz](https://graphviz.org/) for visualization.
an output. Underneath we default to use `graphviz <https://graphviz.org/>`_ for visualization.
Visualize just execution required to create outputs
***************************************************
Expand Down
4 changes: 2 additions & 2 deletions docs/reference/api-reference/available-decorators.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Available Decorators

While the 1:1 mapping of column -> function implementation is powerful, we've implemented a few decorators to promote
business-logic reuse. The decorators we've defined are as follows (source can be found in
`function\_modifiers <https://github.com/stitchfix/hamilton/blob/main/hamilton/function\_modifiers.py>`_):
`function_modifiers <https://github.com/stitchfix/hamilton/blob/main/hamilton/function\_modifiers.py>`_):

@tag
----
Expand Down Expand Up @@ -184,7 +184,7 @@ of the series, and one that checks whether the data is in a certain range.

Note that you can also specify custom decorators using the ``@check_output_custom`` decorator.

See `data\_quality <https://github.com/stitchfix/hamilton/blob/main/data\_quality.md>`_ for more information on
See `data_quality <https://github.com/stitchfix/hamilton/blob/main/data\_quality.md>`_ for more information on
available validators and how to build custom ones.

@parameterize
Expand Down
8 changes: 4 additions & 4 deletions docs/talks-or-podcasts-or-blogs-or-papers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This page curates talks, podcasts, and blogs about Hamilton
- Info / Links
* - August 2022
- Hamilton: a modular open source declarative paradigm for high level modeling of dataflows
- `Paper <https://cdmsworkshop.github.io/2022/Proceedings/ShortPapers/Paper6\_StefanKrawczyk.pdf>`_ at `CDMS Workshop <https://cdmsworkshop.github.io/2022/overview.html>`_ @ `VLDB2022 <https://vldb.org/2022/>`_
- `Paper <https://cdmsworkshop.github.io/2022/Proceedings/ShortPapers/Paper6\_StefanKrawczyk.pdf>`_ at `CDMS Workshop <https://cdmsworkshop.github.io/2022/overview.html>`_ @ `VLDB2022 <https://vldb.org/2022>`_
* - July 2022
- Hamilton: A Python Micro-Framework for Data / Feature Engineering @MLOPs World Bay Area
- `Slides <https://github.com/skrawcz/talks/files/9213924/Hamilton\_.A.Python.Micro-Framework.for.Data.\_.Feature.Engineering.pdf>`_
Expand All @@ -35,14 +35,14 @@ This page curates talks, podcasts, and blogs about Hamilton
- Functions and DAGs: Hamilton, a General Purpose Micro-Framework for Pandas Dataframe Generation
- `Blog post <https://towardsdatascience.com/functions-dags-introducing-hamilton-a-microframework-for-dataframe-generation-more-8e34b84efc1d>`__
* - February 2022
- [Open Source] Hamilton, a micro framework for creating dataframes, and its application at Stitch Fix @ `Apply(meetup) <https://www.applyconf.com/agenda/open-source-hamilton-a-micro-framework-for-creating-dataframes-and-its-application-at-stitch-fix/>`_
- [Open Source] Hamilton, a micro framework for creating dataframes, and its application at Stitch Fix @ `Apply(meetup) <https://www.applyconf.com/agenda/open-source-hamilton-a-micro-framework-for-creating-dataframes-and-its-application-at-stitch-fix>`_
- 30 minute talk about Hamilton, it's origin, tips on using it, and some exciting extensions & plans - `Youtube <https://www.youtube.com/watch?v=CHfrT5OVjlM>`_. For slides, see below.
* - October 2022
- Functions & DAGs: introducing Hamilton, a microframework for dataframe generation
- Blog post on Hamilton origins. `https://multithreaded.stitchfix.com/blog/2021/10/14/functions-dags-hamilton/ <https://multithreaded.stitchfix.com/blog/2021/10/14/functions-dags-hamilton/>`_
- Blog post on Hamilton origins. `Blog post <https://multithreaded.stitchfix.com/blog/2021/10/14/functions-dags-hamilton>`_
* - April 2021
- Hamilton: a Micro Framework for Creating Dataframes @ applyconf
- 10 minute lightning talk on Hamilton `https://www.youtube.com/watch?v=B5Zp\_30Knoo <https://www.youtube.com/watch?v=B5Zp\_30Knoo>`_
- 10 minute lightning talk on Hamilton `Youtube video <https://www.youtube.com/watch?v=B5Zp\_30Knoo>`_

Slides
------
Expand Down
7 changes: 3 additions & 4 deletions docs/tutorials/data-quality-checks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,7 @@ This is easy with the ``check_output`` decorator!
return avg_3wk_spend / signups
It takes in a series of arguments -- you can discover these by exploring the code (more documentation of specific
arguments to follow): `https://github.com/stitchfix/hamilton/blob/main/hamilton/data\_quality/default\_validators.py <https://github.com/stitchfix/hamilton/blob/main/hamilton/data\_quality/default\_validators.py>`_.

arguments to follow): `DefaultValidation <https://github.com/stitchfix/hamilton/blob/main/hamilton/data\_quality/default\_validators.py/>`_
You can also utilize ``pandera`` to the same effect, simply by providing a ``schema`` argument:

.. code-block:: python
Expand Down Expand Up @@ -185,6 +184,6 @@ enabling you to programmatically react to them down the line.
This concludes our brief lessons on managing data quality in Hamilton -- we hope you spin it up and give it a try.

In the mean time, we have more detailed documentation here -
`https://github.com/stitchfix/hamilton/blob/main/data\_quality.md <https://github.com/stitchfix/hamilton/blob/main/data\_quality.md>`_
`data_quality.md <https://github.com/stitchfix/hamilton/blob/main/data\_quality.md/>`_
and some more examples (including distributed systems support) -
`https://github.com/stitchfix/hamilton/tree/main/examples/data\_quality <https://github.com/stitchfix/hamilton/tree/main/examples/data\_quality>`_.
`data_quality examples <https://github.com/stitchfix/hamilton/tree/main/examples/data\_quality/>`_.
8 changes: 4 additions & 4 deletions docs/tutorials/ray.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ For those eager to just see code, please jump right ahead to a full hello world

.. image:: ../_static/Hamilton_ray_post_image.png

For those unfamiliar with `Ray <https://ray.io/>`_, it is an open source framework that can scale python applications
that came out of `UC Berkeley <https://rise.cs.berkeley.edu/projects/ray/>`_. It has a growing ecosystem of tooling
For those unfamiliar with `Ray <https://ray.io>`_, it is an open source framework that can scale python applications
that came out of `UC Berkeley <https://rise.cs.berkeley.edu/projects/ray>`_. It has a growing ecosystem of tooling
that helps with lots of machine learning related workflows. For example, it sells itself as enabling you to scale from
your laptop to a cluster very easily, without having to change much code. In terms of real world use, we like to use
Ray as a very quick way to implement `multiprocessing in python <https://machinelearningmastery.com/multiprocessing-in-python/>`_
Ray as a very quick way to implement `multiprocessing in python <https://machinelearningmastery.com/multiprocessing-in-python>`_
without worry about the details!

Ray Primer
Expand Down Expand Up @@ -151,7 +151,7 @@ been very stable (hasn’t changed since launch), but to feel good about making
think.
#. We don’t expose all the functionality of Ray, but we could. E.g. memory aware scheduling, or specifying resources for
specific functions. Let us know if you want something exposed — create an issue on github please — 
`https://github.com/stitchfix/hamilton <https://github.com/stitchfix/hamilton>`_.
`stitchfix/hamilton <https://github.com/stitchfix/hamilton>`_.

To conclude
-----------
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/using-hamilton-in-a-notebook.rst
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ Pro-tip: You can use ipython magic to autoreload code
-----------------------------------------------------

Open a Python module and a Jupyter notebook side-to-side, and then add
`%autoreload ipython magic <https://ipython.org/ipython-doc/3/config/extensions/autoreload.html>`_ to the notebook to
`autoreload ipython magic <https://ipython.org/ipython-doc/3/config/extensions/autoreload.html>`_ to the notebook to
auto-reload the cell:

.. code-block:: python
Expand Down

0 comments on commit f0ffb4c

Please sign in to comment.