[Feature] Add DMCP and fix the deploy pipeline of NAS algorithms #406

Lxtccc · 2022-12-22T07:44:58Z

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Please describe the motivation of this PR and the goal you want to achieve through this PR.

Modification

Please briefly describe what modification is made in this PR.

BC-breaking (Optional)

Does the modification introduce changes that break the backward compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here and update the documentation.

Checklist

Before PR:

Pre-commit or other linting tools are used to fix the potential lint issues.
Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
CLA has been signed and all committers have signed the CLA in this PR.

sunnyxiaohu · 2022-12-23T06:11:47Z

mmrazor/models/algorithms/pruning/dmcp.py

+            # To static ops for loaded pruned network.
+            self._deploy()
+
+    def _deploy(self):


_deploy is not needed here.

sunnyxiaohu · 2022-12-23T06:13:48Z

mmrazor/models/algorithms/pruning/dmcp.py

+ForwardResults = Union[LossResults, TensorResults, PredictResults]
+
+@MODELS.register_module()
+class DMCP(ItePruneAlgorithm):


inherit from ItePruneAlgorithm for ?

sunnyxiaohu · 2022-12-23T06:26:20Z

mmrazor/models/mutators/channel_mutator/dmcp_channel_mutator.py

+    def __init__(self,
+                 channel_unit_cfg: Union[dict, Type[ChannelUnitType]] = dict(
+                     type='DMCPChannelUnit', units={}),
+                 parse_cfg=dict(


update parse_cfg to ChannelAnalyzer, according to:

mmrazor/mmrazor/models/mutators/channel_mutator/channel_mutator.py

Line 73 in 26da4d2

parse_cfg: Dict = dict(

sunnyxiaohu · 2022-12-23T06:35:35Z

mmrazor/models/mutators/channel_mutator/dmcp_channel_mutator.py

+            else:
+                raise NotImplementedError
+
+    def calc_current_flops(self, model):


move to Algorithm.

sunnyxiaohu · 2022-12-23T06:40:57Z

mmrazor/models/task_modules/estimators/counters/flops_params_counter.py

@@ -172,7 +171,8 @@ def params_units_convert(num_params, units='M', precision=3):
        >>> params_units_convert(3e-9)
        '3e-09'
    """
-
+    if getattr(num_params, 'requires_grad', None):


If you want to get num_params without units, just pass the argument units with the None.

sunnyxiaohu · 2022-12-23T06:49:35Z

mmrazor/models/architectures/dynamic_ops/bricks/dynamic_norm.py

-class DynamicSyncBatchNorm(nn.SyncBatchNorm, DynamicBatchNormMixin):
-    """DynamicOp for sync bn."""
+@MODELS.register_module()
+class DMCPBatchNorm2d(DynamicBatchNorm2d):


fix dynamic_norm.py

Lxtccc · 2022-12-23T09:05:07Z

mmrazor/models/mutators/channel_mutator/dmcp_channel_mutator.py

+
+        self.modify_supernet_forward(arch_train)
+
+    def _prune_by_arch(self, mode, group_id):


Lxtccc · 2022-12-23T09:16:35Z

mmrazor/models/mutators/channel_mutator/dmcp_channel_mutator.py

+            group_arch_param = self._build_arch_param(arch_message[1])
+            self.arch_params[str(group_id)] = group_arch_param
+
+            for m in module[0].output_related:


modify name:m->unit

Lxtccc · 2022-12-23T09:17:44Z

mmrazor/models/mutables/mutable_channel/units/dmcp_channel_unit.py

+                 min_ratio: float = 0.5) -> None:
+        super().__init__(num_channels, choice_mode, divisor, min_value,
+                         min_ratio)
+        self._traceable_choice = tracable_choice()


codecov · 2023-01-11T02:13:02Z

Codecov Report

Patch coverage: 75.92% and project coverage change: +0.35 🎉

Comparison is base (60469c0) 79.14% compared to head (7d57599) 79.49%.

Additional details and impacted files

@@             Coverage Diff             @@
##           dev-1.x     #406      +/-   ##
===========================================
+ Coverage    79.14%   79.49%   +0.35%     
===========================================
  Files          276      279       +3     
  Lines        13597    14027     +430     
  Branches      2084     2155      +71     
===========================================
+ Hits         10761    11151     +390     
- Misses        2395     2416      +21     
- Partials       441      460      +19

Flag	Coverage Δ
unittests	`79.49% <75.92%> (+0.35%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmrazor/engine/hooks/dump_subnet_hook.py	`35.00% <0.00%> (ø)`
mmrazor/engine/runner/subnet_val_loop.py	`24.00% <0.00%> (ø)`
...architectures/backbones/searchable_mobilenet_v3.py	`89.11% <ø> (ø)`
...dynamic_ops/bricks/dynamic_multi_head_attention.py	`69.73% <ø> (ø)`
mmrazor/models/mutables/__init__.py	`100.00% <ø> (ø)`
mmrazor/models/mutators/__init__.py	`100.00% <ø> (ø)`
mmrazor/models/algorithms/nas/bignas.py	`38.28% <10.66%> (+1.05%)`	⬆️
.../models/task_modules/predictor/metric_predictor.py	`65.62% <16.66%> (-7.32%)`	⬇️
...zor/models/architectures/utils/mutable_register.py	`75.75% <28.57%> (-14.57%)`	⬇️
mmrazor/engine/hooks/dmcp_subnet_hook.py	`30.55% <30.55%> (ø)`
... and 27 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

configs/nas/mmcls/bignas/ATTENTIVE_SUBNET_A0.yaml

configs/nas/mmcls/onceforall/ofa_mobilenet_subnet_8xb256_in1k.py

mmrazor/structures/subnet/fix_subnet.py

mmrazor/engine/runner/evolution_search_loop.py

configs/nas/mmcls/autoformer/autoformer_subnet_8xb256_in1k.py

vansin · 2023-03-08T14:39:48Z

@Lxtccc Congratulations on having your first PR merged in OpenMMLab! I am a technical operations staff member of OpenMMLab. My WeChat ID is van-sin ,please add me . I would like to invite you to join the OpenMMLab contributors WeChat group.

openmmlab-bot · 2023-03-30T07:27:58Z

Dear Lxtccc,
First of all, we want to express our gratitude for your significant PR in the MMRazor project. Your contribution is highly appreciated, and we are grateful for your efforts in helping improve this open-source project during your personal time. We believe that many developers will benefit from your PR.
We are looking forward to continuing our collaboration with you. OpenMMLab has established a special contributors' organization called MMSIG, which provides contributors with open-source certificates, a recognition system, and exclusive rewards. You can contact us by adding our WeChat（if you have WeChat): openmmlabwx, or join in our discord：https://discord.gg/raweFPmdzG. We sincerely hope you will join us!
Best regards！ @Lxtccc

Lxtccc added 4 commits December 22, 2022 15:09

Copybook

6bd286b

Newly created copy PR

6fc9f78

Newly created copy PR

55d96c3

update op_counters

26da4d2

sunnyxiaohu approved these changes Dec 23, 2022

View reviewed changes

sunnyxiaohu self-requested a review December 23, 2022 06:42

sunnyxiaohu reviewed Dec 23, 2022

View reviewed changes

Lxtccc commented Dec 23, 2022

View reviewed changes

sunnyxiaohu self-requested a review December 28, 2022 07:24

Lxtccc added 16 commits January 3, 2023 02:35

update subnet/commit/FLOPsCounter

0058373

update docs/UT

f4adf90

update docs/UT

f16ba55

add setter for current_mask

864d08c

replace current_mask with activated_tensor_channel

b41acc3

update subnet training

41b540c

fix ci

5991046

fix ci

c6a01c3

fix ci

8c4d8ea

fix readme.md

bc4bd39

fix readme.md

406cdbc

fix conflict

7830ed9

update

20b2bcb

fix expression

60cc3ce

update

cf81f88

fix CI

c9715c0

Lxtccc force-pushed the DMCP branch from 13fb30a to c9715c0 Compare January 10, 2023 07:32

fix UT

fb7f671

Lxtccc added 2 commits February 17, 2023 18:22

update url

7a72392

Merge branch 'DMCP' of https://github.com/Lxtccc/mmrazor into DMCP

e6eda89

aptsunny reviewed Feb 17, 2023

View reviewed changes

configs/nas/mmcls/bignas/ATTENTIVE_SUBNET_A0.yaml Outdated Show resolved Hide resolved

sunyue1 and others added 4 commits February 23, 2023 14:53

fix a0 fine_grained

d239f1d

fix subnet export misskey

cc1acb1

fix ofa yaml

e0fabc3

fix lint

5510cf8

gaoyang07 reviewed Feb 28, 2023

View reviewed changes

configs/nas/mmcls/onceforall/ofa_mobilenet_subnet_8xb256_in1k.py Outdated Show resolved Hide resolved

mmrazor/structures/subnet/fix_subnet.py Outdated Show resolved Hide resolved

mmrazor/engine/runner/evolution_search_loop.py Outdated Show resolved Hide resolved

gaoyang07 reviewed Feb 28, 2023

View reviewed changes

configs/nas/mmcls/autoformer/autoformer_subnet_8xb256_in1k.py Outdated Show resolved Hide resolved

gaoyang07 and others added 10 commits March 1, 2023 15:30

fix comments

31a062e

merge dev-1.x into dmcp

e9c2c9f

add autoformer cfg

450c885

update readme

0efa384

fix error

a4cbf32

update supernet link

b950750

fix sub_model configs

e98846d

update subnet inference readme

ca39542

fix lint

64a0589

Merge branch 'dev-1.x' into DMCP

c1bc1b9

aptsunny approved these changes Mar 2, 2023

View reviewed changes

aptsunny and others added 5 commits March 2, 2023 09:29

fix lint

7a45886

Update autoformer_subnet_8xb256_in1k.py

83cec1c

update test.py to support args.checkpoint as none

86b50fa

update DARTS readme

573fa42

update readme

7d57599

gaoyang07 force-pushed the DMCP branch from e0e8f39 to 7d57599 Compare March 2, 2023 09:47

sunnyxiaohu changed the title ~~[Feature] Add DMCP~~ [Feature] Add DMCP and fix the deploy pipeline of NAS algorithms Mar 2, 2023

sunnyxiaohu merged commit 0b24276 into open-mmlab:dev-1.x Mar 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Add DMCP and fix the deploy pipeline of NAS algorithms #406

[Feature] Add DMCP and fix the deploy pipeline of NAS algorithms #406

Lxtccc commented Dec 22, 2022

sunnyxiaohu Dec 23, 2022

sunnyxiaohu Dec 23, 2022

sunnyxiaohu Dec 23, 2022

sunnyxiaohu Dec 23, 2022

sunnyxiaohu Dec 23, 2022

sunnyxiaohu Dec 23, 2022

Lxtccc Dec 23, 2022

Lxtccc Dec 23, 2022

Lxtccc Dec 23, 2022

codecov bot commented Jan 11, 2023 •

edited

Loading

vansin commented Mar 8, 2023

openmmlab-bot commented Mar 30, 2023


		self.modify_supernet_forward(arch_train)

		def _prune_by_arch(self, mode, group_id):

[Feature] Add DMCP and fix the deploy pipeline of NAS algorithms #406

[Feature] Add DMCP and fix the deploy pipeline of NAS algorithms #406

Conversation

Lxtccc commented Dec 22, 2022

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jan 11, 2023 • edited Loading

Codecov Report

vansin commented Mar 8, 2023

openmmlab-bot commented Mar 30, 2023

codecov bot commented Jan 11, 2023 •

edited

Loading