[Model Compression] auto compression #3631

J-shang · 2021-05-11T11:33:31Z

Depend on #3507

~~AutoCompressExperiment / AutoCompressionExperiment ?~~
~~AutoCompressExperiment(auto_compress_module, config, training_service) / experiment.config.module_file_path = '...' ?~~

linbinskn · 2021-05-15T06:15:16Z

nni/algorithms/compression/pytorch/auto_compress/auto_compress_engine.py

+            config_dict[(quant_types, op_types, op_names)][var_name] = value
+
+        config_list = []
+        for key, config in config_dict.items():


If quant_bits is set to {'quant_bits':{'weight':8, 'output':8}} in initial quantizer config, the converted config here which is added into config_list would not contain the key quant_bits. Is it still correct?

it contains quant_bits, but I find this kind of nested search space has a bug, I will fix it.

linbinskn · 2021-05-15T07:01:53Z

nni/algorithms/compression/pytorch/auto_compress/interface.py

+
+    @classmethod
+    @abstractmethod
+    def finetune_trainer(cls, compressor_type: str, algorithm_name: str) -> Optional[Callable[[Module, Optimizer], None]]:


What is the difference between trainer and finetune_trainer?

Some pruners need a trainer to help in pruning process, I don't know if it has a difference with the trainer in finetune, so I reserve an interface here. If this is not necessary, I will remove it.

linbinskn · 2021-05-15T07:08:43Z

nni/algorithms/compression/pytorch/auto_compress/auto_compress_engine.py

+        config_list = cls._convert_config_list(compressor_type, converted_config_dict)
+
+        model, evaluator, optimizer = auto_compress_module.model(), auto_compress_module.evaluator(), auto_compress_module.optimizer()
+        trainer = auto_compress_module.trainer(compressor_type, algorithm_name)


Just curious. I haven't found the implementation of trainer in example. So in the shown case, we only finetune model without training it during compress pipeline? Or they represent the same thing so we can ignore 'trainer'.

ultmaster · 2021-05-17T08:17:05Z

nni/algorithms/compression/pytorch/auto_compress/auto_compress_engine.py

+from torch.optim import Optimizer
+
+import nni
+from nni.retiarii.utils import import_


I'd rather maintain another copy of import_ here than import from retiarii.

Importing from another component looks weird and un-self-contained.

ultmaster · 2021-05-17T08:19:52Z

examples/model_compress/auto_compress/torch/auto_compress_module.py

+    _test(_model)
+    _scheduler.step()
+
+class AutoCompressModule(AbstractAutoCompressModule):


Where is this module used?

This module is implemented by user, and will import by import_ in AutoCompressEngine.trial_execute_compress().

It is strange to fix the code file name auto_compress_module.py, I will modify this.

do users have to use the name "AutoCompressModule"?

refactor and no need to fix name AutoCompressModule.

please add docstring for the member functions

QuanluZhang · 2021-05-25T02:25:25Z

docs/en_US/Compression/AutoCompression.rst

+The main differences are as follows:
+
+* Use a generator to help generate search space object.
+* Need to implement the abstract class ``AbstractAutoCompressModule`` as ``AutoCompressModule``.


what is the meaning of this line?

modified to be more readable.

QuanluZhang · 2021-05-25T02:30:24Z

docs/en_US/Compression/AutoCompression.rst

+        'bnn': BNNQuantizer
+    }
+
+Implement ``AbstractAutoCompressModule``


-> Provide user model

QuanluZhang · 2021-05-25T02:31:00Z

docs/en_US/Compression/AutoCompression.rst

+Implement ``AbstractAutoCompressModule``
+----------------------------------------
+
+This class will be called by ``AutoCompressEngine`` on training service.


do not mention this at the beginning

QuanluZhang · 2021-05-25T02:36:10Z

docs/en_US/Compression/AutoCompression.rst

+
+Similar to launch from python, the difference is no need to set ``trial_command``.
+By default, ``auto_compress_module_file_name`` is set as ``./auto_compress_module.py``.
+Remember that ``auto_compress_module_file_name`` is the relative file path under ``trial_code_directory``.


it has to be relative path, why?

no need anymore

QuanluZhang · 2021-05-25T02:38:08Z

docs/en_US/Compression/AutoCompression.rst

+    experiment.config.training_service.use_active_gpu = True
+
+    # the relative file path under trial_code_directory, which contains the class AutoCompressModule
+    experiment.config.auto_compress_module_file_name = './auto_compress_module.py'


better to put this config as AutoCompressExperiment's input argument

refactor it

QuanluZhang · 2021-05-25T02:39:06Z

docs/en_US/Compression/AutoPruningUsingTuners.rst

@@ -3,7 +3,7 @@ Automatic Model Pruning using NNI Tuners

 It's convenient to implement auto model pruning with NNI compression and NNI tuners

-First, model compression with NNI
+First, model pruning with NNI


can directly remove this file

QuanluZhang · 2021-05-25T02:39:22Z

docs/en_US/Compression/advanced.rst

@@ -6,4 +6,5 @@ Advanced Usage

    Framework <./Framework>
    Customize a new algorithm <./CustomizeCompressor>
-    Automatic Model Compression <./AutoPruningUsingTuners>
+    Automatic Model Pruning <./AutoPruningUsingTuners>


QuanluZhang · 2021-05-25T02:42:59Z

examples/model_compress/auto_compress/torch/auto_compress_module.py

+        return _model
+
+    @classmethod
+    def optimizer(cls) -> torch.optim.Optimizer:


seems you do not mention optimizer in doc? do users need to implement this function?

rewritten the doc and mention optimizer and other interfaces.

QuanluZhang · 2021-05-25T03:11:34Z

nni/algorithms/compression/pytorch/auto_compress/utils.py

+                '_type': 'choice',
+                '_value': compressor_choice_value
+            }
+        }


it is better to merge pruner_choice_list and quantizer_choice_list together, then the nest depth is reduced by one, which is simpler and is friendly to tuning algorithms

merge these.

QuanluZhang · 2021-05-25T03:21:03Z

nni/algorithms/compression/pytorch/auto_compress/auto_compress_engine.py

+        return model
+
+    @classmethod
+    def __compress_quantization_pipeline(cls, algorithm_name: str,


the two functions can be merged together

it has a little different, pls see the new update, I think it’s clearer to implement separately.

QuanluZhang · 2021-05-25T03:22:17Z

nni/algorithms/compression/pytorch/auto_compress/auto_compress_engine.py

+        return model
+
+    @classmethod
+    def _compress_pipeline(cls, compressor_type: str,


why it is called "pipeline"?

renaming _compress_pipeline() -> _compress()

QuanluZhang · 2021-05-25T05:39:35Z

nni/algorithms/compression/pytorch/auto_compress/experiment.py

+            raise AttributeError(f'{key} is not supposed to be set in AutoCompress mode by users!')
+        # 'trial_code_directory' is handled differently because the path will be converted to absolute path by us
+        if key == 'trial_code_directory' and not (value == Path('.') or os.path.isabs(value)):
+            raise AttributeError(f'{key} is not supposed to be set in AutoCompress mode by users!')


this is not a correct error message...

and why the value must be Path('.') or absolute path?

have removed this.

QuanluZhang · 2021-05-25T07:46:33Z

docs/en_US/Compression/AutoCompression.rst

+    from pathlib import Path
+    from nni.algorithms.compression.pytorch.auto_compress import AutoCompressExperiment
+
+    experiment = AutoCompressExperiment('local')


exp = AutoCompressExperiment('local', AutoCompressModule)

refactor it.

linbinskn · 2021-05-25T14:06:47Z

docs/en_US/Compression/AutoCompression.rst

+    experiment.config.trial_code_directory = Path(__file__).parent
+    experiment.config.tuner.name = 'TPE'
+    experiment.config.tuner.class_args['optimize_mode'] = 'maximize'
+    experiment.config.training_service.use_active_gpu = True


If I am not mistaken, this feature is for users who want to try different model compression algorithms without many effort. I think some of they would be confused about the experiment config setting if they are not familiar with NNI. Maybe we should tell user what these experiment parameters are or refer to related NNI doc which introduces parameters in detail.

Good suggestion, trying to refactor and use the original config for less effort.

Refactor and now we can use experiment = AutoCompressExperiment(AutoCompressModule, 'local'), no need to use a specific config.

linbinskn · 2021-05-25T14:34:27Z

docs/en_US/Compression/AutoCompression.rst

+
+This approach is mainly a combination of compression and nni experiments.
+It allows users to define compressor search space, including types, parameters, etc.
+Its using experience is similar to launch the NNI experiment from python.


Just personal concern, maybe not correct. I think this doc mainly focuses on how to use this new feature, but it doesn't tell users what things this feature can actually help them do clearly and users will be confused or misunderstand.
In my opinion, this feature can help users try different compression algorithms including pruning algorithms and quantization algorithms by adding them into our 'search space'. By using it, users can easily choose different compression algorithms and apply them to model to get feedback easily and automatically. But If I am a brand new user, after reading this doc, I can't get this key point and still miss some important information such as

what 'search space', 'types' and parameters mean?

what is the meaning of 'auto compress'?

what is the meaning of 'combination of compression and nni experiments'?

If I want to try different compression algorithms, will they be applied together(apply pruning algorithm and quantization to model simultaneously) or single sequentially?

Good suggestion, will add more descriptions and explanations about auto compress and what this can help.

Rewrite the doc and welcome more comments.

QuanluZhang · 2021-05-27T02:52:44Z

docs/en_US/Compression/AutoCompression.rst

+The main differences are as follows:
+
+* Use a generator to help generate search space object.
+* Need to provide the model to be compressed, and the model should have already pre-trained.


have already been pre-trained

QuanluZhang · 2021-05-27T12:07:59Z

Depend on #3507

AutoCompressExperiment / AutoCompressionExperiment ?

~~AutoCompressExperiment(auto_compress_module, config, training_service) / experiment.config.module_file_path = '...' ?~~

AutoCompressionExperiment is better

J-shang · 2021-05-27T12:42:17Z

Depend on #3507

AutoCompressExperiment / AutoCompressionExperiment ?

~~AutoCompressExperiment(auto_compress_module, config, training_service) / experiment.config.module_file_path = '...' ?~~

AutoCompressionExperiment is better

Rename AutoCompressXXX to AutoCompressionXXX

linbinskn · 2021-05-27T14:10:46Z

docs/en_US/Compression/AutoCompression.rst

+Generate search space
+---------------------
+
+Due to the extensive use of nested search space, we recommend a using generator to configure search space.


recommend using a?

J-shang added 3 commits May 11, 2021 19:31

add auto compress

668ad4b

add example

9ae11f7

update

070111b

ultmaster linked an issue May 12, 2021 that may be closed by this pull request

Multi and Auto Compressor #3584

Closed

J-shang added 2 commits May 12, 2021 10:36

update

b882f0d

rename

d15f49d

ultmaster requested review from linbinskn and QuanluZhang May 14, 2021 08:41

ultmaster added the model compression label May 14, 2021

ultmaster self-requested a review May 14, 2021 08:52

linbinskn reviewed May 15, 2021

View reviewed changes

fix nested convert

03d21cf

ultmaster reviewed May 17, 2021

View reviewed changes

J-shang added 2 commits May 17, 2021 16:47

handle import_

dea696f

update config auto compress module file path

a85abc7

J-shang mentioned this pull request May 18, 2021

Abstract class for automatic compression #3390

Closed

J-shang added 2 commits May 18, 2021 14:45

add doc

8544847

update comments

8ef3e0d

J-shang marked this pull request as ready for review May 18, 2021 06:47

QuanluZhang reviewed May 25, 2021

View reviewed changes

QuanluZhang requested review from colorjam and QuanluZhang May 25, 2021 02:25

QuanluZhang reviewed May 25, 2021

View reviewed changes

J-shang and others added 2 commits May 25, 2021 17:02

Merge branch 'microsoft:master' into auto-compressor

5f185a4

update doc

cc85d47

linbinskn reviewed May 25, 2021

View reviewed changes

J-shang added 3 commits May 26, 2021 03:15

merge compress type in search space & modify AutoCompressExperiment api

c6ee7a3

update doc

627ec1c

fix lint

ea692b1

J-shang requested review from ultmaster, QuanluZhang and linbinskn May 26, 2021 07:05

ultmaster approved these changes May 27, 2021

View reviewed changes

QuanluZhang reviewed May 27, 2021

View reviewed changes

update doc & example

ddc3c06

J-shang added 2 commits May 27, 2021 20:38

rename autocompress to autocompression

6339a53

rename

75af835

J-shang changed the title ~~[Model Compression] auto compress~~ [Model Compression] auto compression May 27, 2021

J-shang requested a review from QuanluZhang May 27, 2021 12:42

linbinskn reviewed May 27, 2021

View reviewed changes

linbinskn approved these changes May 27, 2021

View reviewed changes

QuanluZhang approved these changes May 28, 2021

View reviewed changes

ultmaster merged commit a8879dd into microsoft:master May 28, 2021

ultmaster mentioned this pull request May 31, 2021

NNI 2021 May~June Iteration Planning #3581

Closed

49 tasks

[Model Compression] auto compression #3631

[Model Compression] auto compression #3631

Conversation

J-shang commented May 11, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linbinskn May 25, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

QuanluZhang commented May 27, 2021

J-shang commented May 27, 2021

Choose a reason for hiding this comment

J-shang commented May 11, 2021 •

edited

Loading

linbinskn May 25, 2021 •

edited

Loading