Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOCS-2653: Fix links in Modin's documentation #2654

Merged
merged 2 commits into from
Jan 29, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions docs/UsingSQLonRay/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Our plans with the SQL API for Modin are to create an interface that allows you
intermix SQL and pandas operations without copying the entire dataset into a new
structure between the two. This is possible due to the architecture of Modin. Currently,
Modin has a query compiler that acts as an intermediate layer between the query language
(e.g. SQL, pandas) and the execution (See architecture_ documentation for details).
(e.g. SQL, pandas) and the execution (See :doc:`architecture </developer/architecture>` documentation for details).

*We have implemented a simple example that can be found below. Feedback welcome!*

Expand All @@ -29,5 +29,3 @@ Modin has a query compiler that acts as an intermediate layer between the query
col1 col2 column 3 col4
0 1 2.0 A String of information True
1 6 17.0 A String of different information False

.. _architecture: https://modin.readthedocs.io/en/latest/developer/architecture.html
12 changes: 5 additions & 7 deletions docs/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ If you're interested in getting involved in the development of Modin, but aren't
where start, take a look at the issues tagged `Good first issue`_ or Documentation_.
These are issues that would be good for getting familiar with the codebase and better
understanding some of the more complex components of the architecture. There is
documentation here about the architecture_ that you will want to review in order to get
started.
documentation here about the :doc:`architecture </developer/architecture>` that you will
YarShev marked this conversation as resolved.
Show resolved Hide resolved
want to review in order to get started.

Also, feel free to join the discussions on the `developer mailing list`_.

Expand Down Expand Up @@ -143,7 +143,7 @@ Adding a test

If you find yourself fixing a bug or adding a new feature, don't forget to add a test to
the test suite to verify its correctness! More on testing and the layout of the tests
can be found in our testing_ documentation. We ask that you follow the existing
can be found in our testing documentation. We ask that you follow the existing
structure of the tests for ease of maintenance.

Running the tests
Expand Down Expand Up @@ -214,6 +214,7 @@ Some details about using Modin on Ray with Asv:
Some details for maintenance:

* ``modin/asv_bench/asv.conf.json`` contains the modin dependencies, with which testing takes place.

YarShev marked this conversation as resolved.
Show resolved Hide resolved
We need to keep them up to date according to the dependencies in ``setup.py``.


Expand Down Expand Up @@ -241,7 +242,7 @@ Contributing a new execution framework or in-memory format
----------------------------------------------------------

If you are interested in contributing support for a new execution framework or in-memory
format, please make sure you understand the architecture_ of Modin.
format, please make sure you understand the :doc:`architecture </developer/architecture>` of Modin.

The best place to start the discussion for adding a new execution framework or in-memory
format is the `developer mailing list`_.
Expand All @@ -250,11 +251,8 @@ More docs on this coming soon...

.. _Good first issue: https://github.com/modin-project/modin/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue+%3Abeginner%3A%22
.. _Documentation: https://github.com/modin-project/modin/issues?q=is%3Aissue+is%3Aopen+label%3A%22documentation+%3Abookmark_tabs%3A%22
.. _architecture: architecture.html
.. _internal methods:
.. _black: https://github.com/ambv/black
.. _flake8: http://flake8.pycqa.org/en/latest/
.. _Github Actions: https://github.com/features/actions
.. _testing:
YarShev marked this conversation as resolved.
Show resolved Hide resolved
.. _Asv: https://github.com/airspeed-velocity/asv#airspeed-velocity
.. _developer mailing list: https://groups.google.com/forum/#!forum/modin-dev
14 changes: 5 additions & 9 deletions docs/developer/architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -234,27 +234,23 @@ Supported Execution Frameworks and Memory Formats

This is the list of execution frameworks and memory formats supported in Modin. If you
would like to contribute a new execution framework or memory format, please see the
documentation page on :doc:`../contributing`.
documentation page on :doc:`contributing </contributing>`.

- `Pandas on Ray`_
- :doc:`Pandas on Ray </UsingPandasonRay/index>`
- Uses the Ray_ execution framework.
- The compute kernel/in-memory format is a pandas DataFrame.
- `Pandas on Dask`_
- :doc:`Pandas on Dask </UsingPandasonDask/index>`
- Uses the `Dask Futures`_ execution framework.
- The compute kernel/in-memory format is a pandas DataFrame.
- `Pyarrow on Ray`_ (experimental)
- :doc:`Pyarrow on Ray </UsingPyarrowonRay/index>` (experimental)
- Uses the Ray_ execution framework.
- The compute kernel/in-memory format is a pyarrow Table.

.. _pandas Dataframe: https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.html
.. _pandas Dataframe: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
.. _Arrow tables: https://arrow.apache.org/docs/python/generated/pyarrow.Table.html
.. _Ray: https://github.com/ray-project/ray
.. _code: https://github.com/modin-project/modin/blob/master/modin/engines/base/frame/data.py
.. _Contributing: contributing.html
YarShev marked this conversation as resolved.
Show resolved Hide resolved
.. _Pandas on Ray: UsingPandasonRay/optimizations.html
.. _Pandas on Dask: UsingPandasonDask/optimizations.html
.. _Dask Futures: https://docs.dask.org/en/latest/futures.html
.. _issue: https://github.com/modin-project/modin/issues
.. _Discourse: https://discuss.modin.org
.. _task parallel: https://en.wikipedia.org/wiki/Task_parallelism
.. _Pyarrow on Ray: UsingPyarrowonRay/index.html
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,6 @@ nature, you get a fast DataFrame at 1MB and 1TB+.
troubleshooting
contact

.. _Dataframe: https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.html
.. _Dataframe: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
.. _Ray: https://github.com/ray-project/ray/
.. _Dask: https://dask.org/
4 changes: 1 addition & 3 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ You may already have a recent version of Dask_ installed, in which case you can
Building Modin from Source
--------------------------

If you're planning on contributing_ to Modin, you will need to ensure that you are
If you're planning on :doc:`contributing </contributing>` to Modin, you will need to ensure that you are
YarShev marked this conversation as resolved.
Show resolved Hide resolved
building Modin from the local repository that you are working off of. Occasionally,
there are issues in overlapping Modin installs from pypi and from source. To avoid these
issues, we recommend uninstalling Modin before you install from source:
Expand All @@ -109,8 +109,6 @@ Once cloned, ``cd`` into the ``modin`` directory and use ``pip`` to install:

.. _`GitHub repo`: https://github.com/modin-project/modin/tree/master
.. _issue: https://github.com/modin-project/modin/issues
.. _`out of core`: out_of_core.html
.. _WSL: https://docs.microsoft.com/en-us/windows/wsl/install-win10
.. _Ray: http://ray.readthedocs.io
.. _contributing: contributing.html
.. _Dask: https://github.com/dask/dask
5 changes: 2 additions & 3 deletions docs/out_of_core.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Install Modin out of core
-------------------------

Modin now comes with all the dependencies for out of core functionality by default! See
the `installation page`_ for more information on installing Modin.
the :doc:`installation page </installation>` for more information on installing Modin.

Starting Modin with out of core enabled
---------------------------------------
Expand Down Expand Up @@ -62,5 +62,4 @@ This example creates a 40GB DataFrame from 20 identical 2GB DataFrames and perfo
various operations on them. Feel free to play around with this code and let us know what
you think!

.. _Dataframe: https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.html
.. _`installation page`: installation.html
.. _Dataframe: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
17 changes: 6 additions & 11 deletions docs/supported_apis/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ For your convenience, we have compiled a list of currently implemented APIs and
available in Modin. This documentation is updated as new methods and APIs are merged
into the master branch, and not necessarily correct as of the most recent release. In
order to install the latest version of Modin, follow the directions found on the
`installation page`_.
:doc:`installation page </installation>`.

Questions on implementation details
-----------------------------------
Expand Down Expand Up @@ -33,20 +33,15 @@ Modin.

The exact methods we have implemented are listed in the respective subsections:

* DataFrame_
* Series_
* utilities_
* `I/O`_
* :doc:`DataFrame </supported_apis/dataframe_supported>`
* :doc:`Series </supported_apis/series_supported>`
* :doc:`utilities </supported_apis/utilities_supported>`
* :doc:`I/O </supported_apis/io_supported>`

We have taken a community-driven approach to implementing new methods. We did a `study
on pandas usage`_ to learn what the most-used APIs are. Modin currently supports **93%**
of the pandas API based on our study of pandas usage, and we are actively expanding the
API.

.. _DataFrame: dataframe_supported.html
.. _Series: series_supported.html
.. _utilities: utilities_supported.html
.. _I/O: io_supported.html
.. _study on pandas usage: https://github.com/modin-project/study_kaggle_usage
YarShev marked this conversation as resolved.
Show resolved Hide resolved
.. _`developer mailing list`: https://groups.google.com/forum/#!forum/modin-dev
.. _`installation page`: installation.html#building-modin-from-source
.. _`study on pandas usage`: https://github.com/modin-project/study_kaggle_usage
3 changes: 1 addition & 2 deletions docs/supported_apis/series_supported.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The second column is a flag for whether or not there is an implementation in Mod
the method in the left column. ``Y`` stands for yes, ``N`` stands for no, ``P`` stands
for partial (meaning some parameters may not be supported yet), and ``D`` stands for
default to pandas. To learn more about the implementations that default to pandas, see
the related section on `Defaulting to pandas`_.
the related section on :doc:`Defaulting to pandas </supported_apis/index>`.

+-----------------------------+---------------------------------+----------------------------------------------------+
| Series method | Modin Implementation? (Y/N/P/D) | Notes for Current implementation |
Expand Down Expand Up @@ -487,4 +487,3 @@ the related section on `Defaulting to pandas`_.
+-----------------------------+---------------------------------+----------------------------------------------------+

.. _`GitHub repository`: https://github.com/modin-project/modin/issues
.. _`Defaulting to pandas`: index.html
11 changes: 4 additions & 7 deletions docs/using_modin.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Using Modin on a Single Node
**In local (without a cluster) modin will create and manage a local (dask or ray) cluster for the execution**

In order to use the most up-to-date version of Modin, please follow the instructions on
the `installation page`_
the :doc:`installation page </installation>`.

Once you import the library, you should see something similar to the following output:

Expand Down Expand Up @@ -52,7 +52,7 @@ Please note, the API is not yet complete. For some methods, you may see the foll

NotImplementedError: To contribute to Modin, please visit github.com/modin-project/modin.

We have compiled a list of `currently supported methods`_.
We have compiled a list of :doc:`currently supported methods </supported_apis/index>`.

If you would like to request a particular method be implemented, feel free to `open an
issue`_. Before you open an issue please make sure that someone else has not already
Expand Down Expand Up @@ -90,7 +90,7 @@ you can customize your Ray environment for use in Modin!
Exceeding memory (Out of core pandas)
"""""""""""""""""""""""""""""""""""""

Modin experimentally supports out of core operations. See more on the `Out of Core`_
Modin experimentally supports out of core operations. See more on the :doc:`Out of Core </out_of_core>`
documentation page.

Reducing or limiting the resources Modin can use
Expand Down Expand Up @@ -134,13 +134,10 @@ Examples
You can find an example on our recent `blog post`_ or on the `Jupyter Notebook`_ that we
used to create the blog post.

.. _`DataFrame`: https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.html
.. _`DataFrame`: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
.. _`pandas`: https://pandas.pydata.org/pandas-docs/stable/
.. _`installation page`: https://modin.readthedocs.io/en/latest/installation.html
.. _`currently supported methods`: https://modin.readthedocs.io/en/latest/pandas_supported.html
.. _`open an issue`: https://github.com/modin-project/modin/issues
.. _`autoscaler documentation`: https://ray.readthedocs.io/en/latest/autoscaling.html
.. _`Ray's documentation`: https://ray.readthedocs.io/en/latest/api.html
.. _`blog post`: https://rise.cs.berkeley.edu/blog/pandas-on-ray-early-lessons/
.. _`Jupyter Notebook`: https://gist.github.com/devin-petersohn/f424d9fb5579a96507c709a36d487f24#file-pandas_on_ray_blog_post_0-ipynb
.. _`Out of Core`: out_of_core.html