[microNPU] Introduce a pass to remove redundant identity operations #10254

lhutton1 · 2022-02-15T13:01:42Z

Introduces a pass that aims to remove identity operations, introduced during legalization, that are immediately followed by an NPU compute operation e.g. Convolution.

cc @manupa-arm @ekalda @mbaret @NicolaLancellotti @jacobbohlin @dchauhan-arm

ekalda

Thanks for the patch, looks really good so far! :) Maybe one testcase to add (to the already very comprehensive set) would be to check that there are no reshape or strided slice directly followed by concatenate.

tests/python/contrib/test_ethosu/test_identity_optimizer.py

lhutton1 · 2022-02-15T17:47:52Z

Thanks for the review @ekalda, I added a test case and removed the print among a couple more things I spotted :)

ekalda

Thanks @lhutton1, looks good to me! :)

lhutton1 · 2022-02-17T11:55:26Z

So it turns out supporting the removal of identity operations with multiple outputs was not as complicated as I first thought, and it was technically already supported in the original patch. Therefore, CountIdentityOutputs and the restriction in RemoveRedundantIdentities can be removed. Added tests for the multiple output case to support this. Apologies for the confusion @ekalda, could you take another look when you have time :)

ekalda

Thanks @lhutton1 , looks all good to me! :)

Introduces a pass that aims to remove identity operations, introduced during legalization, that are immediately followed by an NPU compute operation e.g. Convolution. Change-Id: Ia3b2c7bebf8cba1f827af8e3f3335677ba8f6371

Change-Id: Idf9341ce757b849f8819944dab2fb3b1496a2caf

Changes in test_identity_optimizer.py: * Fixed typo in docstring * Removed print * Fixed same output test to use correct input shape Changes in codegen.cc: * Remove unnecessary constructor Change-Id: Ie4a053725110ce52d8be039ca1ce48084bc66545

Change-Id: I0a88d92dd31ca3dd07a2a495f18c10a2ebf2fc9e

Builds upon the work in apache#10254 to remove identity operations sandwiched between two non-compute operations (reshape/strided slice - concatenate is handled differently), under certain conditions. Specifically, an identity operation is not removed when the dimensionality between the two non-compute operations is reduced, due to non-congruent values being accessed incorrectly. For example, ``` strided_slice(dims=4) -> identity -> reshape(dims=4) ``` becomes... ``` strided_slice -> reshape ``` but, ``` strided_slice(dims=4) -> identity -> reshape(dims=2) ``` becomes... ``` strided_slice -> reshape ``` Change-Id: Ie28ba384fcb3230d6f4651c0c19e2b9526ebcc42

Builds upon the work in apache#10254 to remove identity operations sandwiched between two non-compute operations (reshape/strided slice - concatenate is handled differently), under certain conditions. Specifically, an identity operation is not removed when the dimensionality between the two non-compute operations is reduced, due to non-congruent values being accessed incorrectly. For example, ``` strided_slice(dims=4) -> identity -> reshape(dims=4) ``` becomes... ``` strided_slice -> reshape ``` but, ``` strided_slice(dims=4) -> identity -> reshape(dims=2) ``` remains as... ``` strided_slice -> identity -> reshape ``` Change-Id: Ie28ba384fcb3230d6f4651c0c19e2b9526ebcc42

Change-Id: Ib54031fe1c70159728876a23f96b72adb2ea17b0

manupak

LGTM!

manupak · 2022-03-08T16:31:59Z

Thanks @ekalda @lhutton1 !

Builds upon the work in apache#10254 to remove identity operations sandwiched between two non-compute operations (reshape/strided slice - concatenate is handled differently), under certain conditions. Specifically, an identity operation is not removed when the dimensionality between the two non-compute operations is reduced, due to non-congruent values being accessed incorrectly. For example, ``` strided_slice(dims=4) -> identity -> reshape(dims=4) ``` becomes... ``` strided_slice -> reshape ``` but, ``` strided_slice(dims=4) -> identity -> reshape(dims=2) ``` remains as... ``` strided_slice -> identity -> reshape ``` Change-Id: Ie28ba384fcb3230d6f4651c0c19e2b9526ebcc42

…pache#10254) * [microNPU] Introduce a pass to remove redundant identity operations Introduces a pass that aims to remove identity operations, introduced during legalization, that are immediately followed by an NPU compute operation e.g. Convolution. Change-Id: Ia3b2c7bebf8cba1f827af8e3f3335677ba8f6371 * fix lint Change-Id: Idf9341ce757b849f8819944dab2fb3b1496a2caf * Addressing comments Changes in test_identity_optimizer.py: * Fixed typo in docstring * Removed print * Fixed same output test to use correct input shape Changes in codegen.cc: * Remove unnecessary constructor Change-Id: Ie4a053725110ce52d8be039ca1ce48084bc66545 * skip tests when required packages are not available Change-Id: I0a88d92dd31ca3dd07a2a495f18c10a2ebf2fc9e * support multiple output identities and add more tests Change-Id: Ib54031fe1c70159728876a23f96b72adb2ea17b0

Builds upon the work in apache#10254 to remove identity operations sandwiched between two non-compute operations (reshape/strided slice - concatenate is handled differently), under certain conditions. Specifically, an identity operation is not removed when the dimensionality between the two non-compute operations is reduced, due to non-congruent values being accessed incorrectly. For example, ``` strided_slice(dims=4) -> identity -> reshape(dims=4) ``` becomes... ``` strided_slice -> reshape ``` but, ``` strided_slice(dims=4) -> identity -> reshape(dims=2) ``` remains as... ``` strided_slice -> identity -> reshape ``` Change-Id: Ie28ba384fcb3230d6f4651c0c19e2b9526ebcc42

…10411) Builds upon the work in #10254 to remove identity operations sandwiched between two non-compute operations (reshape/strided slice - concatenate is handled differently), under certain conditions. Specifically, an identity operation is not removed when the dimensionality between the two non-compute operations is reduced, due to non-congruent values being accessed incorrectly. For example, ``` strided_slice(dims=4) -> identity -> reshape(dims=4) ``` becomes... ``` strided_slice -> reshape ``` but, ``` strided_slice(dims=4) -> identity -> reshape(dims=2) ``` remains as... ``` strided_slice -> identity -> reshape ``` Change-Id: Ie28ba384fcb3230d6f4651c0c19e2b9526ebcc42

…pache#10254) * [microNPU] Introduce a pass to remove redundant identity operations Introduces a pass that aims to remove identity operations, introduced during legalization, that are immediately followed by an NPU compute operation e.g. Convolution. Change-Id: Ia3b2c7bebf8cba1f827af8e3f3335677ba8f6371 * fix lint Change-Id: Idf9341ce757b849f8819944dab2fb3b1496a2caf * Addressing comments Changes in test_identity_optimizer.py: * Fixed typo in docstring * Removed print * Fixed same output test to use correct input shape Changes in codegen.cc: * Remove unnecessary constructor Change-Id: Ie4a053725110ce52d8be039ca1ce48084bc66545 * skip tests when required packages are not available Change-Id: I0a88d92dd31ca3dd07a2a495f18c10a2ebf2fc9e * support multiple output identities and add more tests Change-Id: Ib54031fe1c70159728876a23f96b72adb2ea17b0

…pache#10411) Builds upon the work in apache#10254 to remove identity operations sandwiched between two non-compute operations (reshape/strided slice - concatenate is handled differently), under certain conditions. Specifically, an identity operation is not removed when the dimensionality between the two non-compute operations is reduced, due to non-congruent values being accessed incorrectly. For example, ``` strided_slice(dims=4) -> identity -> reshape(dims=4) ``` becomes... ``` strided_slice -> reshape ``` but, ``` strided_slice(dims=4) -> identity -> reshape(dims=2) ``` remains as... ``` strided_slice -> identity -> reshape ``` Change-Id: Ie28ba384fcb3230d6f4651c0c19e2b9526ebcc42

lhutton1 requested review from zhiics, trevor-m, comaniac, mbaret, manupak, jroesch, slyubomirsky, icemelon, MarisaKirisame, ZihengJiang, yzhliu, vinx13, mbrookhart, jwfromm, anijain2305, wweic, junrushao, tqchen, areusch and merrymercy as code owners February 15, 2022 13:01

ekalda reviewed Feb 15, 2022

View reviewed changes

tests/python/contrib/test_ethosu/test_identity_optimizer.py Outdated Show resolved Hide resolved

lhutton1 mentioned this pull request Feb 16, 2022

[Tracking Issue] Continued Arm(R) Ethos(TM)-U NPU operator support #9961

Closed

11 tasks

ekalda approved these changes Feb 16, 2022

View reviewed changes

lhutton1 force-pushed the identity-optimizer branch 2 times, most recently from 0db7b03 to 7d2b6c8 Compare February 17, 2022 11:49

ekalda approved these changes Feb 24, 2022

View reviewed changes

lhutton1 added 2 commits February 28, 2022 14:23

fix lint

e8ffea9

Change-Id: Idf9341ce757b849f8819944dab2fb3b1496a2caf

lhutton1 added 2 commits February 28, 2022 14:23

Addressing comments

4b1eb5e

Changes in test_identity_optimizer.py: * Fixed typo in docstring * Removed print * Fixed same output test to use correct input shape Changes in codegen.cc: * Remove unnecessary constructor Change-Id: Ie4a053725110ce52d8be039ca1ce48084bc66545

skip tests when required packages are not available

6e760db

Change-Id: I0a88d92dd31ca3dd07a2a495f18c10a2ebf2fc9e

lhutton1 mentioned this pull request Feb 28, 2022

[microNPU] Remove identity operations between non-compute operations #10411

Merged

support multiple output identities and add more tests

31fff76

Change-Id: Ib54031fe1c70159728876a23f96b72adb2ea17b0

lhutton1 force-pushed the identity-optimizer branch from 7d2b6c8 to 31fff76 Compare March 2, 2022 17:41

manupak approved these changes Mar 8, 2022

View reviewed changes

manupak merged commit 5e81389 into apache:main Mar 8, 2022

lhutton1 deleted the identity-optimizer branch March 9, 2022 10:13

driazati mentioned this pull request Jul 14, 2022

TVM v0.9.0.rc0 Release Candidate Notes #12102

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[microNPU] Introduce a pass to remove redundant identity operations #10254

[microNPU] Introduce a pass to remove redundant identity operations #10254

lhutton1 commented Feb 15, 2022

ekalda left a comment

lhutton1 commented Feb 15, 2022

ekalda left a comment

lhutton1 commented Feb 17, 2022

ekalda left a comment

manupak left a comment

manupak commented Mar 8, 2022

[microNPU] Introduce a pass to remove redundant identity operations #10254

[microNPU] Introduce a pass to remove redundant identity operations #10254

Conversation

lhutton1 commented Feb 15, 2022

ekalda left a comment

Choose a reason for hiding this comment

lhutton1 commented Feb 15, 2022

ekalda left a comment

Choose a reason for hiding this comment

lhutton1 commented Feb 17, 2022

ekalda left a comment

Choose a reason for hiding this comment

manupak left a comment

Choose a reason for hiding this comment

manupak commented Mar 8, 2022