Refactor flops counter #3048

colorjam · 2020-10-30T08:27:01Z

Format each layer info for easy debug
Solve multiple calls of one module

QuanluZhang · 2020-11-04T05:31:12Z

@colorjam please resolve conflict

…r-dev

chicm-ms · 2020-11-09T06:15:39Z

docs/en_US/Compression/CompressionUtils.md

+
+# Given input tensor with size (1, 1, 28, 28) and switch to full mode
+x = torch.randn(1, 1, 28, 28)
+flops, params, results = count_flops_params(model, (x, ), mode='full') 


The parameter (x,) seems not consistent with code. should be x ?

Thanks~ fixed it, please review the latest version~

chicm-ms · 2020-11-09T06:31:14Z

nni/compression/pytorch/utils/counter.py

+        "module_type": type(m).__name__
+    }
+
+    add_results(m._name, **results)


How about count_xxx functions just return the results, and do the add_results outside? it will make count_xxx functions cleaner.

we can use a common hook for register_forward_hook and call count_xxx per module type inside the common hook

Thanks. I have updated the code, please review the latest version~

countxxx still calls self._push_results, I mean define another common hook function for example myhook to call countxxx by module type, then do _push_results in myhook. Register my hook as the hook using something like m.register_forward_hook(myhook), in this way countxxx is a pure util function with single responsibility and can be reused later.

QuanluZhang · 2020-11-10T01:32:37Z

docs/en_US/Compression/CompressionUtils.md

@@ -121,14 +121,19 @@ fixed_mask = fix_mask_conflict('./resnet18_mask', net, data)
 ```

 ## Model FLOPs/Parameters Counter
-We provide a model counter for calculating the model FLOPs and parameters. This counter supports calculating FLOPs/parameters of a normal model without masks, it can also calculates FLOPs/parameters of a model with mask wrappers, which helps users easily check model complexity during model compression on NNI. Note that, for sturctured pruning, we only identify the remained filters according to its mask, which not taking the pruned input channels into consideration, so the calculated FLOPs will be larger than real number (i.e., the number calculated after Model Speedup).
+We provide a model counter for calculating the model FLOPs and parameters. This counter supports calculating FLOPs/parameters of a normal model without masks, it can also calculates FLOPs/parameters of a model with mask wrappers, which helps users easily check model complexity during model compression on NNI. Note that, for sturctured pruning, we only identify the remained filters according to its mask, which not taking the pruned input channels into consideration, so the calculated FLOPs will be larger than real number (i.e., the number calculated after Model Speedup). We support two modes to collect information of modules. The first mode is `default`, which only collect the information of convolution and linear. The second mode is `full`, which also collect the information of other operations. Users can easily use our collected `results` for futher analysis.


use a new paragraph for your added content

nni/compression/pytorch/utils/counter.py

QuanluZhang · 2020-11-10T02:59:08Z

nni/compression/pytorch/utils/counter.py

-    input_size: list, tuple
-        the input shape of data
+    x: tuple or tensor
+        the input shape of data or a tensor as input data
    custom_ops: dict
        a mapping of (module: custom operation)


it is not clear what is the type of "custom operation", it is a function? the key is module type?

yes, user should manually define a function for counting flops of the module

QuanluZhang · 2020-11-10T03:00:09Z

nni/compression/pytorch/utils/counter.py


    Returns
    -------
    flops: float
        total flops of the model
    params:
        total params of the model
+    results: dict


only write type here, for example, dict

QuanluZhang · 2020-11-10T03:02:14Z

nni/compression/pytorch/utils/counter.py


    Returns
    -------
    flops: float
        total flops of the model
    params:
        total params of the model
+    results: dict
+        detail information of modules


what kind of information?

QuanluZhang · 2020-11-10T03:10:31Z

nni/compression/pytorch/utils/counter.py


-    device = next(model.parameters()).device
-    inputs = torch.randn(input_size).to(device)
+    if torch.is_tensor(x[0]):


so here x is a list?

yes, x are positional arguments given to the module

docs/en_US/Compression/CompressionUtils.md

QuanluZhang · 2020-11-12T02:22:40Z

nni/compression/pytorch/utils/counter.py

-    assert input_size is not None
+class ModelProfiler:
+    # use a class to share state to hooks
+    # profile results are available in `self.results`


please use standard docstring format

This is a private implementation class. Not visible to user (not in __all__) anyway.

ultmaster · 2020-11-12T02:35:41Z

nni/compression/pytorch/utils/counter.py

+    Count FLOPs and Params of the given model. This function would identify the mask on the module
+    and take the pruned shape into consideration. Note that, for sturctured pruning, we only identify
+    the remained filters according to its mask, and do not take the pruned input channels into consideration,
+    so the calculated FLOPs will be larger than real number.


Add one more blank line here too.

ultmaster · 2020-11-12T02:35:47Z

nni/compression/pytorch/utils/counter.py

+        the mode of how to collect information. If the mode is set to `default`,
+        only the information of convolution and linear will be collected.
+        If the mode is set to `full`, other operations will also be collected.
+    Returns


QuanluZhang · 2020-11-17T09:28:53Z

nni/compression/pytorch/utils/counter.py

+
+        if len(list(m.children())) == 0 and type(m) in profiler.ops:
+            # if a leaf node
+            _handler = m.register_forward_hook(functools.partial(profiler.counte_module, name=name))


counte_module -> count_module

colorjam added 2 commits October 16, 2020 17:39

update title level

1817648

support layer info

5bb7527

liuzhe-lz mentioned this pull request Oct 30, 2020

v2.0 Release Plan #2935

Closed

77 tasks

colorjam added 5 commits November 2, 2020 17:26

update

b939e0c

Merge branch 'master' into counter-dev

d290d20

update

ea6050a

Merge branch 'master' of https://github.com/microsoft/nni

8c639c4

Merge branch 'master' into counter-dev

edf0938

liuzhe-lz requested review from chicm-ms and QuanluZhang November 4, 2020 02:46

Merge branch 'master' of https://github.com/microsoft/nni into counte…

828729a

…r-dev

colorjam changed the base branch from v2.0 to master November 4, 2020 06:35

colorjam and others added 9 commits November 4, 2020 15:38

delete files

f8e0d2d

delete files

136d274

format result

6a4f520

update

464adae

remove ut in counter

0ab3ed6

fix pipeline

bb76bce

fix pipeline

d9a0a45

fix pipeline

5187d1c

fix pipeline

53e7b75

chicm-ms reviewed Nov 9, 2020

View reviewed changes

QuanluZhang reviewed Nov 10, 2020

View reviewed changes

nni/compression/pytorch/utils/counter.py Outdated Show resolved Hide resolved

QuanluZhang reviewed Nov 10, 2020

View reviewed changes

update

d4483e4

QuanluZhang reviewed Nov 12, 2020

View reviewed changes

docs/en_US/Compression/CompressionUtils.md Show resolved Hide resolved

QuanluZhang reviewed Nov 12, 2020

View reviewed changes

ultmaster reviewed Nov 12, 2020

View reviewed changes

colorjam added 6 commits November 12, 2020 12:07

pretty docstring

40da63e

add blank line

5bcdead

remove whitespace

a39276e

remove whitespace

59cdf84

update

852907d

add line

7534117

ultmaster approved these changes Nov 12, 2020

View reviewed changes

colorjam added 6 commits November 13, 2020 17:57

Fix ut

e41f2b7

Remove ut in counter

3558fa1

Update up

675d292

Use single module counter

ba7770d

Fix minus typo

78fae9d

Fix minus typo

9099310

QuanluZhang reviewed Nov 17, 2020

View reviewed changes

chicm-ms approved these changes Nov 17, 2020

View reviewed changes

colorjam and others added 2 commits November 18, 2020 02:33

Add get result function

7fd24a7

Fix docstring and a few other typos

472f051

QuanluZhang approved these changes Nov 17, 2020

View reviewed changes

colorjam added 3 commits November 18, 2020 09:40

Fix pylint

05eac2b

Fix pipeline

2c3cd16

Remove whitespace

82ab8e0

liuzhe-lz closed this Nov 20, 2020

liuzhe-lz reopened this Nov 20, 2020

liuzhe-lz merged commit b6233e5 into microsoft:master Nov 23, 2020

linbinskn mentioned this pull request Jan 5, 2021

fix flops counter bug in auto_pruners_torch.py #3264

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor flops counter #3048

Refactor flops counter #3048

colorjam commented Oct 30, 2020 •

edited

Loading

QuanluZhang commented Nov 4, 2020

chicm-ms Nov 9, 2020

colorjam Nov 10, 2020

chicm-ms Nov 9, 2020 •

edited

Loading

colorjam Nov 10, 2020

chicm-ms Nov 11, 2020

colorjam Nov 18, 2020

QuanluZhang Nov 10, 2020

colorjam Nov 10, 2020

QuanluZhang Nov 10, 2020

colorjam Nov 10, 2020

QuanluZhang Nov 10, 2020

QuanluZhang Nov 10, 2020

QuanluZhang Nov 10, 2020

colorjam Nov 10, 2020

QuanluZhang Nov 12, 2020

ultmaster Nov 12, 2020

ultmaster Nov 12, 2020

colorjam Nov 16, 2020

ultmaster Nov 12, 2020

colorjam Nov 16, 2020

QuanluZhang Nov 17, 2020

Refactor flops counter #3048

Refactor flops counter #3048

Conversation

colorjam commented Oct 30, 2020 • edited Loading

QuanluZhang commented Nov 4, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chicm-ms Nov 9, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

colorjam commented Oct 30, 2020 •

edited

Loading

chicm-ms Nov 9, 2020 •

edited

Loading