Skip to content

Commit

Permalink
DOCS-#2518: add asv usage topic (#2549)
Browse files Browse the repository at this point in the history
* DOCS-#2518: add asv usage topic

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* DOCS-#2518: fix style

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>

* DOCS-#2518: address review comments

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
  • Loading branch information
anmyachev authored Dec 17, 2020
1 parent a58dcf6 commit ab29ed6
Showing 1 changed file with 53 additions and 0 deletions.
53 changes: 53 additions & 0 deletions docs/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,58 @@ subset of the test suite. In order to run a specific test run:
The entire test suite is automatically run for each pull request.

Performance measurement
-----------------------

We use Asv_ tool for performance tracking of various Modin functionality.

Here are some scenarios in which Asv can be used:

* It is necessary to check the impact of the new patch on the performance of a certain set of operations:

.. code-block:: bash
asv continuous -f 1.05 src/master HEAD -b TimeGroupBy --launch-method=spawn
* It is necessary to check presence of errors inside of benchmarks after making changes or writing new ones:

.. code-block:: bash
asv run --quick --show-stderr --python=same --launch-method=spawn
* You just need to run the entire test suite to get the current time numbers:

.. code-block:: bash
asv run --launch-method=spawn
* It is necessary to check the range of commits for performance degradation:

.. code-block:: bash
asv run [start_hash]..[end_hash] --launch-method=spawn
asv publish
asv preview
For more consistent results, you may need to use the following parameters:

* ``-a sample_time=1``
* ``-a warmup_time=1``
* ``-a processes=4``

Some details about using Modin on Ray with Asv:

* ``--launch-method=forkserver`` is not working
* Each set of parameters for each test is launched in its own process, which brings
a large overhead, since for each process redis server and other necessary binaries
from ray initialization are started and destroyed.

Some details for maintenance:

* ``modin/asv_bench/asv.conf.json`` contains the modin dependencies, with which testing takes place.
We need to keep them up to date according to the dependencies in ``setup.py``.


Building documentation
----------------------

Expand Down Expand Up @@ -204,4 +256,5 @@ More docs on this coming soon...
.. _flake8: http://flake8.pycqa.org/en/latest/
.. _Github Actions: https://github.com/features/actions
.. _testing:
.. _Asv: https://github.com/airspeed-velocity/asv#airspeed-velocity
.. _developer mailing list: https://groups.google.com/forum/#!forum/modin-dev

0 comments on commit ab29ed6

Please sign in to comment.