| Travis | Jenkins |
|---|---|
 |
- It integrates multiple repos, including onnx, pytorch, caffe2, to provide a convenient environment for developers.
- OnnxBot automatically updates the submodules in
reposfolder, runs the tests, and automatically detects the breaking changes from different repos.
./install.sh # or ./install-develop.sh to install the most of the repos in develop mode.
./test.shDockerfile may be helpful for setting up the whole repo.
repos: submodulestest: test scriptsjenkins: jenkins CI configurationtravis: travis CI configuration
This repo contains several test scripts in test folder.
test_caffe2.py: PyTorch ==> ONNX ==> Caffe2 end-to-end tests, in which the decimal precision is checked.test_models.py: PyTorch ==> ONNX model tests, check the output ONNX graph with expected files.test_operators.py: PyTorch ==> ONNX operator tests, check the output ONNX graph with expected files.test_verify.py: PyTorch ==> ONNX verification tests, check the exporting raises exceptions in certain conditions.test_pytorch_helper.py: PyTorch ==> ONNX helper tests, check the pytorch related helper functions.
To update the expected files, please add flag --accept to the end of the command.
Using convert_pytorch_test.py, we can easily turn some existing PyTorch testing cases to ONNX backend test cases. Just run:
python test/convert_pytorch_test.pyScript test_operators.py is also able to generate ONNX backend test cases based on operator tests. Just run:
python test/test_operators.py --onnx-testAll the generated ONNX backend test data is stored in repos/onnx/onnx/backend/test/data/model.
But the generated test cases may not pass end-to-end tests, so we have another script filter_generated_test.py
to check whether the cases are passed or not, and the failed data will be moved to
test/failed/generated.
filter_generated_test.py supports the following flags:
-v: verbose--delete: delete failed test cases, by default, they are moved totest/failed/generated.--no-expect: do not generate expect files for each test, by default, expected files are generated intest/expectfolder for debugging purpose.