-
Notifications
You must be signed in to change notification settings - Fork 6.8k
[MXNET-34] Onnx Module to import onnx models into mxnet #9963
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The underscore prefix in the name of the folder _import
makes it look like an internal module. Any good reason of naming it in this way?
For all the test cases to pass for this module, we will need changes in the CI process. It needs Onnx and protobuf in the CI process for the tests to run. Currently the onnx-mxnet project fetches these packages in the CI process - [Container] 2018/03/02 00:14:07 Running command wget https://github.com/google/protobuf/releases/download/v2.6.1/protobuf-2.6.1.tar.gz
Downloading onnx-1.0.1.tar.gz (763kB) |
Hi @anirudhacharya, are you blocked by the CI change? You are able to change the CI process along with your PR, and installing two dependencies shouldn't be hard. Just let us know if you'd like help doing this. (To get started you can see how other python deps are installed here: https://github.com/apache/incubator-mxnet/blob/master/tests/ci_build/Dockerfile.cpu and here: https://github.com/apache/incubator-mxnet/blob/master/tests/ci_build/install/install_testdeps.sh for example.) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the contribution.
Few minor comments.
python/mxnet/contrib/onnx/_import/translation_utils/__init__.py
Outdated
Show resolved
Hide resolved
python/mxnet/contrib/onnx/_import/op_translations/reduce_max.py
Outdated
Show resolved
Hide resolved
@piiswrong can you please take a look at the code. We will keep adding more operator support, but can you review the general design of the module. I think I have addressed all your concerns. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Can you please check PR build failures.
Hi, the community has passed to vote about associating the code changes with JIRA (https://lists.apache.org/thread.html/ab22cf0e35f1bce2c3bf3bec2bc5b85a9583a3fe7fd56ba1bbade55f@%3Cdev.mxnet.apache.org%3E) We have updated the guidelines for contributors in https://cwiki.apache.org/confluence/display/MXNET/Development+Process, please ensure that you have created a JIRA at https://issues.apache.org/jira/projects/MXNET/issues/ to describe your work in this pull request and include the JIRA title in your PR as [MXNET-xxxx] your title where MXNET-xxxx is the JIRA id Thanks! |
* Change package name to onnx from serde. * Remove onnx install time dependency * Remove Renamer class * Add apache license to files. * Refactor test files to tests/python folder. * Removed export folder. * Refactor Attribute COnverter logic Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com>
Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com>
c04881d
to
0212292
Compare
Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two minor changes
Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a correctness test
Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved in terms of CI
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, except for a few comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, lets merge after the CI tests are run.
Is there a plan to add a doc page with these changes? Normally we don't expect users to find out APIs by reading the code. |
Will add it as a separate PR as per review comments. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com>
@reminisce there is a follow up PR with the doc page changes. Please refer - #10140 |
* Onnx Module to import onnx models into mxnet * Change package name to onnx from serde. * Remove onnx install time dependency * Remove Renamer class * Add apache license to files. * Refactor test files to tests/python folder. * Removed export folder. * Refactor Attribute COnverter logic Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Changing the translation and utils file. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * - Fixed Pylint issues - Added Sigmoid operator. - Add onnx, protobuf as CI pipeline dependencies. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Add UTs for reduce ops. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * pylint - newline, whitespace. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Added operators: - AvgPool - ArgMax - ArgMin - Abs Minor changes in logic for import_onnx * Added operators: - Ceil - Cast - Constant * - Added Pad operator support. - Minor changes for comments * RandomUniform,Normal,Sub,Mul,Div,Tanh,Relu,Reciprocal,Sqrt operators added. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * lint fix * Add protobuf-compile to CI bash script. Add MatMul and Pow operator. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Max,Min,Sum,Reduce operators. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * BatchNorm,SpatialBN, Split Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Slice,Transpose and Squeeze Operators. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Onnx tests in CI integration tests. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Addressing Marco's comments Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Floor, LeakyRelu, Elu, PRelu, Softmax, Exp, Log operator. * Added operators: - Convolution - Deconvolution Refactored convert_operator * lint fix * Rebase fix * Added Maxpool operator * Adding FullyConnected operator * Adding operator- GlobalPooling - max and avg Minor lint fixes. * Adding operator - Gemm * Change test Path, LRN and Dropout operator. * Add asserts for the super_res example. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Fixing conv test failures. Removed redundant code * Update Jenkins job. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Nits: Removing commented out code * Rebase after Docker PR * Fetch test files by version number. Verify the high resolution example. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Fix method arguments for Python3.5+ Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Remove logging configuration from test files. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Verify result image in example by hash Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Remove fetching test files by ETag. Will add it as a separate PR as per review comments. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com>
* Onnx Module to import onnx models into mxnet * Change package name to onnx from serde. * Remove onnx install time dependency * Remove Renamer class * Add apache license to files. * Refactor test files to tests/python folder. * Removed export folder. * Refactor Attribute COnverter logic Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Changing the translation and utils file. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * - Fixed Pylint issues - Added Sigmoid operator. - Add onnx, protobuf as CI pipeline dependencies. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Add UTs for reduce ops. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * pylint - newline, whitespace. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Added operators: - AvgPool - ArgMax - ArgMin - Abs Minor changes in logic for import_onnx * Added operators: - Ceil - Cast - Constant * - Added Pad operator support. - Minor changes for comments * RandomUniform,Normal,Sub,Mul,Div,Tanh,Relu,Reciprocal,Sqrt operators added. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * lint fix * Add protobuf-compile to CI bash script. Add MatMul and Pow operator. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Max,Min,Sum,Reduce operators. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * BatchNorm,SpatialBN, Split Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Slice,Transpose and Squeeze Operators. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Onnx tests in CI integration tests. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Addressing Marco's comments Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Floor, LeakyRelu, Elu, PRelu, Softmax, Exp, Log operator. * Added operators: - Convolution - Deconvolution Refactored convert_operator * lint fix * Rebase fix * Added Maxpool operator * Adding FullyConnected operator * Adding operator- GlobalPooling - max and avg Minor lint fixes. * Adding operator - Gemm * Change test Path, LRN and Dropout operator. * Add asserts for the super_res example. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Fixing conv test failures. Removed redundant code * Update Jenkins job. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Nits: Removing commented out code * Rebase after Docker PR * Fetch test files by version number. Verify the high resolution example. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Fix method arguments for Python3.5+ Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Remove logging configuration from test files. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Verify result image in example by hash Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Remove fetching test files by ETag. Will add it as a separate PR as per review comments. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com>
* Onnx Module to import onnx models into mxnet * Change package name to onnx from serde. * Remove onnx install time dependency * Remove Renamer class * Add apache license to files. * Refactor test files to tests/python folder. * Removed export folder. * Refactor Attribute COnverter logic Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Changing the translation and utils file. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * - Fixed Pylint issues - Added Sigmoid operator. - Add onnx, protobuf as CI pipeline dependencies. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Add UTs for reduce ops. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * pylint - newline, whitespace. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Added operators: - AvgPool - ArgMax - ArgMin - Abs Minor changes in logic for import_onnx * Added operators: - Ceil - Cast - Constant * - Added Pad operator support. - Minor changes for comments * RandomUniform,Normal,Sub,Mul,Div,Tanh,Relu,Reciprocal,Sqrt operators added. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * lint fix * Add protobuf-compile to CI bash script. Add MatMul and Pow operator. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Max,Min,Sum,Reduce operators. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * BatchNorm,SpatialBN, Split Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Slice,Transpose and Squeeze Operators. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Onnx tests in CI integration tests. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Addressing Marco's comments Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Floor, LeakyRelu, Elu, PRelu, Softmax, Exp, Log operator. * Added operators: - Convolution - Deconvolution Refactored convert_operator * lint fix * Rebase fix * Added Maxpool operator * Adding FullyConnected operator * Adding operator- GlobalPooling - max and avg Minor lint fixes. * Adding operator - Gemm * Change test Path, LRN and Dropout operator. * Add asserts for the super_res example. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Fixing conv test failures. Removed redundant code * Update Jenkins job. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Nits: Removing commented out code * Rebase after Docker PR * Fetch test files by version number. Verify the high resolution example. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Fix method arguments for Python3.5+ Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Remove logging configuration from test files. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Verify result image in example by hash Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Remove fetching test files by ETag. Will add it as a separate PR as per review comments. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com>
* Onnx Module to import onnx models into mxnet * Change package name to onnx from serde. * Remove onnx install time dependency * Remove Renamer class * Add apache license to files. * Refactor test files to tests/python folder. * Removed export folder. * Refactor Attribute COnverter logic Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Changing the translation and utils file. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * - Fixed Pylint issues - Added Sigmoid operator. - Add onnx, protobuf as CI pipeline dependencies. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Add UTs for reduce ops. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * pylint - newline, whitespace. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Added operators: - AvgPool - ArgMax - ArgMin - Abs Minor changes in logic for import_onnx * Added operators: - Ceil - Cast - Constant * - Added Pad operator support. - Minor changes for comments * RandomUniform,Normal,Sub,Mul,Div,Tanh,Relu,Reciprocal,Sqrt operators added. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * lint fix * Add protobuf-compile to CI bash script. Add MatMul and Pow operator. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Max,Min,Sum,Reduce operators. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * BatchNorm,SpatialBN, Split Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Slice,Transpose and Squeeze Operators. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Onnx tests in CI integration tests. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Addressing Marco's comments Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Floor, LeakyRelu, Elu, PRelu, Softmax, Exp, Log operator. * Added operators: - Convolution - Deconvolution Refactored convert_operator * lint fix * Rebase fix * Added Maxpool operator * Adding FullyConnected operator * Adding operator- GlobalPooling - max and avg Minor lint fixes. * Adding operator - Gemm * Change test Path, LRN and Dropout operator. * Add asserts for the super_res example. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Fixing conv test failures. Removed redundant code * Update Jenkins job. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Nits: Removing commented out code * Rebase after Docker PR * Fetch test files by version number. Verify the high resolution example. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Fix method arguments for Python3.5+ Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Remove logging configuration from test files. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Verify result image in example by hash Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com> * Remove fetching test files by ETag. Will add it as a separate PR as per review comments. Signed-off-by: Acharya <aanirud@8c85904b0bf4.ant.amazon.com>
Description
Module to import onnx models into mxnet.
Checklist
Essentials
make lint
)Changes
Comments
@Roshrini @spidydev @lupesko @reminisce @piiswrong @szha
@nswamy @sandeep-krishnamurthy
@awslabs/aws-deep-learning-sdk