Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

moved hamilton-ui docs to top-level #898

Merged
merged 2 commits into from
May 31, 2024
Merged

moved hamilton-ui docs to top-level #898

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a27d055
moved hamilton-ui to top-level
May 8, 2024
afc8618
Adds symlink to UI docs
skrawcz May 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion docs/concepts/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,4 @@ concepts that makes Hamilton unique and powerful.
builder
function-modifiers-advanced
parallel-task
ui
skrawcz marked this conversation as resolved.
Show resolved Hide resolved
best-practices/index
11 changes: 11 additions & 0 deletions docs/hamilton-ui/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
===========
Hamilton UI
===========

Reference
---------

.. toctree::
:maxdepth: 2

ui
110 changes: 45 additions & 65 deletions docs/concepts/ui.rst → docs/hamilton-ui/ui.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
===========
Hamilton UI
===========
-----------
Overview
-----------

Hamilton comes with a fully open-source UI that can be run both for local deployment and on a remote server.
The UI consists of the following features:
Expand All @@ -15,74 +15,62 @@ In short, the Hamilton UI aims to combine a large swath of MLOps/data observabil
.. image:: ../_static/hamilton_ui.jpeg
:alt: Hamilton UI

--------
Overview
--------

If you'd like a video walkthrough on getting set up, you can watch the following:

.. raw:: html
---

<iframe width="560" height="315" src="https://www.youtube.com/embed/DPfxlTwaNsM?si=gks5oOAWsNPSJbe_" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

If you'd like a quick overview of some of the features, you can watch the following:
The Hamilton UI is contained within a set of Docker images. You launch with `docker-compose <https://docs.docker.com/compose/>`_, and it will start up the UI, the backend server,
and a Postgres database. If you'd like a quick overview of some of the features, you can watch the following:

.. raw:: html

<iframe width="560" height="315" src="https://www.youtube.com/embed/0VIVSeN7Ij8?si=i3vTsfTNorzh5y2C" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

Getting Started
---------------

The Hamilton UI is contained within a set of docker images. You launch with `docker-compose <https://docs.docker.com/compose/>`_, and it will start up the UI, the backend server,
and a postgres database.

-------
Install
-------

Prerequisites
-------------
If you'd like a video walkthrough on getting set up, you can watch the following:

To run this, you'll need:
.. raw:: html

1. Docker installed -- you can follow instructions `here <https://docs.docker.com/engine/install/>`_.
2. A Hamilton workflow -- if you don't have this there's an init command that will create a basic one for you.
3. The `hamilton` repository cloned locally.
<iframe width="560" height="315" src="https://www.youtube.com/embed/DPfxlTwaNsM?si=gks5oOAWsNPSJbe_" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

----

Starting the UI
---------------
As prerequisites, you will need to have Docker installed -- you can follow instructions `here <https://docs.docker.com/engine/install/>`_.

Start by ensuring that the hamilton repository is cloned locally. Then, navigate to the `ui` directory:
1. Clone the Hamilton repository locally

.. code-block:: bash

git clone https://github.com/dagworks-inc/hamilton

Then, navigate to the ``ui`` directory:
2. Navigate to the ``hamilton/ui`` directory

.. code-block:: bash

cd hamilton/ui


And run the server command:
3. Execute the installation script with the following command

.. code-block:: bash

./deployment/run.sh


This will do the following:
This will:

- Pull all Docker images from the Docker Hub
- Start a local Postgres database
- Start the backend server
- Start the frontend server

1. Pull all docker images from the docker hub
2. Start a local postgres database
3. Start the backend server
4. Start the frontend server
This takes a bit of time! So be patient. The server will be running on port 8242.

This takes a bit of time! So be patient. The server will be running on port 8242. Then navigate to ``http://localhost:8242`` in your browser, and enter your email (this will be the username used within the app).
4. Then navigate to ``http://localhost:8242`` in your browser, and enter your email (this will be the username used within the app).

Building the Docker Images locally
__________________________________
-----------------------------------
If building the Docker containers from scratch, increase your Docker memory to 10gb or more -- you can do this in the Docker Desktop settings.

To build the images locally, you can run the following command:
Expand All @@ -98,10 +86,15 @@ This will build the containers from scratch. If you just want to mount the local

./deployment/dev.sh

----
Self-Hosting
-------------

Please reach out to us if you want to deploy on your own infrastructure. Self-hosting documentation will be up soon.


Running your first dataflows
----------------------------
-----------
Get started
-----------

Now that you have your server running, you can run a simple dataflow and watch it in the UI!
You can follow instructions in the UI when you create a new project, or follow the instructions here.
Expand All @@ -119,7 +112,7 @@ Then, navigate to the project page (dashboard/projects), in the running UI, and
Remember the project ID -- you'll use it for the next steps.

Existing Hamilton Code
______________________
----------------------
Add the following adapter to your code if you have existing Hamilton code:

.. code-block:: python
Expand All @@ -144,7 +137,7 @@ Add the following adapter to your code if you have existing Hamilton code:
Then run your DAG, and follow the links in the logs!

I need some Hamilton code to run
________________________________
--------------------------------
If you don't have Hamilton code to run this with, you can run Hamilton UI example under `examples/hamilton_ui <https://github.com/dagworks-inc/hamilton/tree/main/examples/hamilton_ui>`_:

.. code-block:: bash
Expand All @@ -158,40 +151,35 @@ If you don't have Hamilton code to run this with, you can run Hamilton UI exampl

You should see links in the `logs to the UI <http://localhost:8242/dashboard/projects>`_, where you can see the DAG run + the data summaries captured.

Exploring the UI
-------------------
----------
Features
----------

Once you get to the UI, you can navigate to the projects page (left hand nav-bar). Assuming you have created a project
and logged to it, you can then navigate to view it and then more details about it. E.g. versions, code, lineage, catalog, execution runs.
See below for a few screenshots of the UI.

-----------
Features
-----------

The UI has the following features:


DAG version tracking
Dataflow versioning
--------------------

Select DAG versions to compare + visualize.
Select a dataflow versions to compare and visualize.

.. image:: ../_static/version_tracking.png
:alt: DAG Version Tracking

Feature/asset Catalog
---------------------
Assets/features catalog
-----------------------

View functions, nodes, and assets across a history of runs.

.. image:: ../_static/catalog.png
:alt: Catalog

Browser
-------
--------

View DAG shapes + code:
View dataflow structure and code.


.. image:: ../_static/code_browser.png
Expand All @@ -201,7 +189,7 @@ View DAG shapes + code:
:alt: Browser

Run tracking + telemetry
------------------------
-------------------------

View a history of runs, telemetry on runs/comparison, and data for specific runs:

Expand All @@ -213,11 +201,3 @@ View a history of runs, telemetry on runs/comparison, and data for specific runs

.. image:: ../_static/run_data.png
:alt: Run Data

----

------------
Self-Hosting
------------

Please reach out to us if you want to deploy on your own infrastructure. Self-hosting documentation will be up soon.
1 change: 1 addition & 0 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
get-started/index
concepts/index
how-tos/index
hamilton-ui/index
integrations/index
code-comparisons/index
```
Expand Down
4 changes: 2 additions & 2 deletions docs/main.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
</div>

Hamilton is a general-purpose framework to write dataflows using regular Python functions. At the core, each function defines a transformation and its parameters indicates its dependencies. Hamilton automatically connects individual functions into a [Directed Acyclic Graph](https://en.wikipedia.org/wiki/Directed_acyclic_graph) (DAG) that can be executed, visualized, optimized, and reported on.
Hamilton also comes with a [UI](concepts/ui.rst) to visualize, catalog, and monitor your dataflows.
Hamilton also comes with a [UI](hamilton-ui/index.rst) to visualize, catalog, and monitor your dataflows.

```{figure} ./_static/abc.png
:scale: 50
Expand All @@ -17,7 +17,7 @@ The ABC of Hamilton

# Why should you use Hamilton?
**Facilitate collaboration**. By focusing on functions, Hamilton avoids sprawling code hierarchy and generates flat dataflows. Well-scoped functions make it easier to add features, complete code reviews, debug pipeline failures, and hand-off projects. Visualizations can be generated directly from your code to better understand and document it.
Integration with the [Hamilton UI](concepts/ui.rst) allows you to track lineage, catalog code & artifacts, and monitor your dataflows.
Integration with the [Hamilton UI](hamilton-ui/index.rst) allows you to track lineage, catalog code & artifacts, and monitor your dataflows.

**Reduce development time**. Hamilton dataflows are reusable across projects and context (e.g., pipeline vs. web service). The benefits of developing robust and well-tested solutions are multiplied by reusability. Off-the-shelf dataflows are available on the [Hamilton Hub](https://hub.dagworks.io/).

Expand Down