Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Refactor Compression Doc #3371

Merged
merged 23 commits into from
Feb 25, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 110 additions & 5 deletions docs/en_US/Compression/CompressionReference.rst
Original file line number Diff line number Diff line change
@@ -1,16 +1,121 @@
Python API Reference of Compression Utilities
=============================================
Model Compression API Reference
===============================

.. contents::

Sensitivity Utilities
Compressors
-----------

Compressor
^^^^^^^^^^
QuanluZhang marked this conversation as resolved.
Show resolved Hide resolved

.. autoclass:: nni.compression.pytorch.compressor.Compressor
:members:

.. autoclass:: nni.compression.pytorch.compressor.Pruner
:members:

.. autoclass:: nni.compression.pytorch.compressor.Quantizer
:members:


Module Wrapper
^^^^^^^^^^^^^^

.. autoclass:: nni.compression.pytorch.compressor.PrunerModuleWrapper
:members:


.. autoclass:: nni.compression.pytorch.compressor.QuantizerModuleWrapper
:members:

Weight Masker
^^^^^^^^^^^^^
.. autoclass:: nni.algorithms.compression.pytorch.pruning.weight_masker.WeightMasker
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.structured_pruning.StructuredWeightMasker
:members:


Pruners
^^^^^^^
.. autoclass:: nni.algorithms.compression.pytorch.pruning.sensitivity_pruner.SensitivityPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.one_shot.OneshotPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.one_shot.LevelPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.one_shot.SlimPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.one_shot.L1FilterPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.one_shot.L2FilterPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.one_shot.FPGMPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.one_shot.TaylorFOWeightFilterPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.one_shot.ActivationAPoZRankFilterPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.one_shot.ActivationMeanRankFilterPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.lottery_ticket.LotteryTicketPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.agp.AGPPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.admm_pruner.ADMMPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.auto_compress_pruner.AutoCompressPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.net_adapt_pruner.NetAdaptPruner
:members:

.. autoclass:: nni.algorithms.compression.pytorch.pruning.simulated_annealing_pruner.SimulatedAnnealingPruner
:members:


Quantizers
^^^^^^^^^^
.. autoclass:: nni.algorithms.compression.pytorch.quantization.quantizers.NaiveQuantizer
:members:

.. autoclass:: nni.algorithms.compression.pytorch.quantization.quantizers.QAT_Quantizer
:members:

.. autoclass:: nni.algorithms.compression.pytorch.quantization.quantizers.DoReFaQuantizer
:members:

.. autoclass:: nni.algorithms.compression.pytorch.quantization.quantizers.BNNQuantizer
:members:



Compression Utilities
---------------------

Sensitivity Utilities
^^^^^^^^^^^^^^^^^^^^^

.. autoclass:: nni.compression.pytorch.utils.sensitivity_analysis.SensitivityAnalysis
:members:

Topology Utilities
------------------
^^^^^^^^^^^^^^^^^^

.. autoclass:: nni.compression.pytorch.utils.shape_dependency.ChannelDependency
:members:
Expand All @@ -28,6 +133,6 @@ Topology Utilities
:members:

Model FLOPs/Parameters Counter
------------------------------
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. autofunction:: nni.compression.pytorch.utils.counter.count_flops_params
12 changes: 4 additions & 8 deletions docs/en_US/Compression/Overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,11 +87,6 @@ Quantization algorithms compress the original network by reducing the number of
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1. `Reference Paper <https://arxiv.org/abs/1602.02830>`__


Automatic Model Compression
---------------------------

Given targeted compression ratio, it is pretty hard to obtain the best compressed ratio in a one shot manner. An automatic model compression algorithm usually need to explore the compression space by compressing different layers with different sparsities. NNI provides such algorithms to free users from specifying sparsity of each layer in a model. Moreover, users could leverage NNI's auto tuning power to automatically compress a model. Detailed document can be found `here <./AutoPruningUsingTuners.rst>`__.

Model Speedup
-------------

Expand All @@ -102,10 +97,11 @@ Compression Utilities

Compression utilities include some useful tools for users to understand and analyze the model they want to compress. For example, users could check sensitivity of each layer to pruning. Users could easily calculate the FLOPs and parameter size of a model. Please refer to `here <./CompressionUtils.rst>`__ for a complete list of compression utilities.

Customize Your Own Compression Algorithms
-----------------------------------------
Advanced Usage
--------------

NNI model compression leaves simple interface for users to customize a new compression algorithm. The design philosophy of the interface is making users focus on the compression logic while hiding framework specific implementation details from users. Users can learn more about our compression framework and customize a new compression algorithm (pruning algorithm or quantization algorithm) based on our framework. Moreover, users could leverage NNI's auto tuning power to automatically compress a model. Please refer to `here <./advanced.rst>`__ for more details.

NNI model compression leaves simple interface for users to customize a new compression algorithm. The design philosophy of the interface is making users focus on the compression logic while hiding framework specific implementation details from users. The detailed tutorial for customizing a new compression algorithm (pruning algorithm or quantization algorithm) can be found `here <./Framework.rst>`__.

Reference and Feedback
----------------------
Expand Down
Loading