Skip to content

Commit

Permalink
Merge pull request #107 from iscoe/cboulay/docs
Browse files Browse the repository at this point in the history 
Big docs update
  • Loading branch information
@griffinmilsap
griffinmilsap committed Apr 16, 2024
2 parents 0e78703 + 3936f67 commit 2682137
Show file tree
Hide file tree
Showing 37 changed files with 1,074 additions and 604 deletions.
242 changes: 43 additions & 199 deletions docs/source/api.rst
View file
Original file line number Diff line number Diff line change
@@ -1,240 +1,84 @@
API
===

An ``ezmsg`` pipeline is created from a few basic components. ``ezmsg`` provides a framework for you to define your own graphs using its building blocks. Inherit from its base components to define a pipeline that works for your project.
An ``ezmsg`` pipeline is created from a few basic components.
``ezmsg`` provides a framework for you to define your own graphs using its building blocks.
Inherit from its base components to define a pipeline that works for your project.

.. automodule:: ezmsg.core

Collection
----------

Connects ``Units`` together by defining a graph which connects ``OutputStreams`` to ``InputStreams``.

Lifecycle Hooks
^^^^^^^^^^^^^^^

The following lifecycle hooks in the ``Collection`` class can be overridden:

.. py:method:: configure( self )
Runs when the ``Collection`` is instantiated. This is the best place to call ``Unit.apply_settings()`` on each member ``Unit`` of the ``Collection``.

Overridable Methods
^^^^^^^^^^^^^^^^^^^^

.. py:method:: network( self ) -> NetworkDefinition
In this method, define and return a ``NetworkDefinition`` which defines how ``InputStreams`` and ``OutputStreams`` from member ``Units`` will be connected.

.. py:method:: process_components( self ) -> Tuple[Unit|Collection, ...]
In this method, define and return a tuple which contains ``Units`` and ``Collections`` which should run in their own processes.

Component
---------

Metaclass which ``Units`` and ``Collections`` inherit from.

Complete
--------

.. py:class:: Complete
A type of ``Exception`` which signals to ``ezmsg`` that the function can be shut down gracefully. If all functions in all ``Units`` raise ``Complete``, the entire pipeline will terminate execution.


NetworkDefinition
------------------

.. py:class:: NetworkDefinition
Wrapper on ``Iterator[Tuple[OutputStream, InputStream]]``.


NormalTermination
-----------------

.. py:class:: NormalTermination
A type of ``Exception`` which signals to ``ezmsg`` that the pipeline can be shut down gracefully.

run
---

.. py:method:: run(components: {str: Component} = None, root_name: str = None, connections: NetworkDefinition = None, process_components: [Component] = None, backend_process: BackendProcess = DefaultBackendProcess, graph_address: (str, int) = None, force_single_process: bool = False, **components_kwargs: Component) -> None
`The old method` run_system() `has been deprecated and uses` run() `instead.`

Begin execution of a set of ``Components``.

`components` represents the nodes in the directed acyclic graph. It is a dictionary which contains the ``Components`` to be run mapped to string names. On initialization, ``ezmsg`` will call ``initialize()`` for each ``Unit`` and ``configure()`` for each ``Collection``, if defined.

`connections` represents the edges is a ``NetworkDefinition`` which connects ``OutputStreams`` to ``InputStreams``. On initialization, ``ezmsg`` will create a directed acyclic graph using the contents of this parameter.

`process_components` is a list of ``Components`` which should live in their own process.

`backend_process` is currently under development.

`graph_address` is a tuple which contains the hostname and port of the graph server which ``ezmsg`` should connect to. If not defined, ``ezmsg`` will start a new graph server at 127.0.0.1:25978.

`force_single_process` will run all ``Components`` in one process. This is necessary when running ``ezmsg`` in a notebook.

Settings
--------

To pass parameters into a ``Component``, inherit from ``Settings``.
Most ``ezmsg`` classes intended for use in building pipelines are available in ``ezmsg.core``.
It is convention to ``import ezmsg.core as ez`` and then use this shorthand in your code. e.g.,

.. code-block:: python
class YourSettings(Settings):
setting1: int
setting2: float
class MyUnit(ez.Unit):
...
To use, declare the ``Settings`` object for a ``Component`` as a member variable called (all-caps!) ``SETTINGS``. ``ezmsg`` will monitor the variable called ``SETTINGS`` in the background, so it is important to name it correctly.
Components
----------

.. code-block:: python
.. autoclass:: Component

class YourUnit(Unit):
.. autoclass:: Collection
:show-inheritance:
:members:

SETTINGS: YourSettings
.. autoclass:: NetworkDefinition

A ``Unit`` can accept a ``Settings`` object as a parameter on instantiation.
.. autoclass:: Unit
:show-inheritance:
:members:
:inherited-members:

.. code-block:: python

class YourCollection(Collection):
Unit Function Decorators
^^^^^^^^^^^^^^^^^^^^^^^^

YOUR_UNIT = YourUnit(
YourSettings(
setting1: int,
setting2: float
)
)
These function decorators can be added to member functions.

.. note::
``Settings`` uses type hints to define member variables, but does not enforce type checking.
.. automethod:: ezmsg.core.subscriber

State
-----
.. automethod:: ezmsg.core.publisher

To track a mutable state for a ``Component``, inherit from ``State``.
.. automethod:: ezmsg.core.main

.. code-block:: python
.. automethod:: unit.thread

class YourState(State):
state1: int
state2: float
.. automethod:: ezmsg.core.task

To use, declare the ``State`` object for a ``Component`` as a member variable called (all-caps!) ``STATE``. ``ezmsg`` will monitor the variable called ``STATE`` in the background, so it is important to name it correctly.
.. automethod:: ezmsg.core.process

Member functions can then access and mutate ``STATE`` as needed during function execution. It is recommended to initialize state values inside the ``initialize()`` or ``configure()`` lifecycle hooks if defaults are not defined.
.. automethod:: ezmsg.core.timeit

.. code-block:: python

class YourUnit(Unit):
Component Interaction
---------------------

STATE: YourState
.. autoclass:: Settings

def initialize(self):
this.STATE.state1 = 0
this.STATE.state2 = 0.0
.. autoclass:: State

.. note::
``State`` uses type hints to define member variables, but does not enforce type checking.

Stream
------

Facilitates a flow of ``Messages`` into or out of a ``Component``.

.. class:: InputStream(Message)

Can be added to any ``Component`` as a member variable. Methods may subscribe to it.


.. class:: OutputStream(Message)

Can be added to any ``Component`` as a member variable. Methods may publish to it.

Facilitates a flow of ``Messages`` into or out of a ``Component``.

Unit
----

Represents a single step in the graph. To create a ``Unit``, inherit from the ``Unit`` class.

Lifecycle Hooks
^^^^^^^^^^^^^^^

The following lifecycle hooks in the ``Unit`` class can be overridden. Both can be run as ``async`` functions by simply adding the ``async`` keyword when overriding.

.. py:method:: initialize( self )
Runs when the ``Unit`` is instantiated.

.. py:method:: shutdown( self )
Runs when the ``Unit`` terminates.

Function Decorators
^^^^^^^^^^^^^^^^^^^

These function decorators can be added to member functions.
.. autoclass:: InputStream

.. py:method:: @subscriber(InputStream)
.. autoclass:: OutputStream

An async function will run once per message received from the ``InputStream`` it subscribes to. Example:

.. code-block:: python
INPUT = ez.InputStream(Message)
@subscriber(INPUT)
async def print_message(self, message: Message) -> None:
print(message)
A function can have both ``@subscriber`` and ``@publisher`` decorators.

.. py:method:: @publisher(OutputStream)
An async function will yield messages on the designated ``OutputStream``.

.. code-block:: python
from typing import AsyncGenerator
OUTPUT = OutputStream(ez.Message)
@publisher(OUTPUT)
async def send_message(self) -> AsyncGenerator:
message = Message()
yield(OUTPUT, message)
A function can have both ``@subscriber`` and ``@publisher`` decorators.

.. py:method:: @main
Designates this function to run as the main thread for this ``Unit``. A ``Unit`` may only have one of these.

.. py:method:: @thread
Designates this function to run as a background thread for this ``Unit``.

.. py:method:: @task
Designates this function to run as a task in the task/messaging thread.

.. py:method:: @process
Designates this function to run in its own process.

.. py:method:: @timeit
Custom Exceptions
-----------------

``ezmsg`` will log the amount of time this function takes to execute.
.. autoclass:: Complete

Public Methods
^^^^^^^^^^^^^^
.. autoclass:: NormalTermination

A class which inherits from ``Unit`` also inherits one public method:

.. function:: Unit.apply_settings( self, settings: Settings )
Entry Point
-----------

Update a ``Unit`` 's ``Settings`` object.
.. automethod:: ezmsg.core.run
38 changes: 36 additions & 2 deletions docs/source/conf.py
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
import os
import sys
import importlib
import inspect

sys.path.insert(0, os.path.abspath('../../src'))
sys.path.insert(0, os.path.abspath('../../extensions/ezmsg-sigproc/src'))
# sys.path.insert(0, os.path.abspath('../../extensions/ezmsg-websocket/src'))
# sys.path.insert(0, os.path.abspath('../../extensions/ezmsg-zmq/src'))

# Configuration file for the Sphinx documentation builder.

# -- Project information
Expand All @@ -6,8 +16,8 @@
copyright = "2022, JHU/APL"
author = "JHU/APL"

release = "3.0"
version = "3.0.0"
release = "3.3.4"
version = "3.3.4"

# -- General configuration

Expand All @@ -17,11 +27,15 @@
"sphinx.ext.autodoc",
"sphinx.ext.autosummary",
"sphinx.ext.intersphinx",
"sphinx.ext.linkcode",
"sphinx.ext.napoleon"
]

intersphinx_mapping = {
"python": ("https://docs.python.org/3/", None),
"sphinx": ("https://www.sphinx-doc.org/en/master/", None),
"numpy": ("https://docs.scipy.org/doc/numpy", None),
"scipy": ("https://docs.scipy.org/doc/scipy/reference", None),
}
intersphinx_disabled_domains = ["std"]

Expand All @@ -33,3 +47,23 @@

# -- Options for EPUB output
epub_show_urls = "footnote"

add_module_names = False


code_url = f"https://github.com/iscoe/ezmsg/blob/dev/"


def linkcode_resolve(domain, info):
if domain != 'py':
return None
if not info['module']:
return None
filename = info['module'].replace('.', '/')
if "sigproc" in filename:
return f"{code_url}extensions/ezmsg-sigproc/src/{filename}.py"
elif "core" in filename:
return f"{code_url}src/ezmsg/core/__init__.py"
else:
return f"{code_url}src/{filename}.py"

7 changes: 6 additions & 1 deletion docs/source/extensions.rst
View file
Original file line number Diff line number Diff line change
@@ -1,2 +1,7 @@
Extensions
==========
==========

.. toctree::
:maxdepth: 1

extensions/sigproc
Loading

0 comments on commit 2682137

Please sign in to comment.