ptq

Evaluate performance of ONNX Runtime(Squeezenet)

ONNX runtime quantization is under active development. please use 1.6.0+ to get more quantization support.

This example load an image classification model from ONNX Model Zoo and confirm its accuracy and speed based on ILSVR2012 validation Imagenet dataset. You need to download this dataset yourself.

Environment

onnx: 1.9.0 onnxruntime: 1.10.0

Prepare model

Download model from ONNX Model Zoo

wget https://github.com/onnx/models/raw/main/vision/classification/squeezenet/model/squeezenet1.0-12.onnx

Quantization

Quantize model with QLinearOps:

bash run_tuning.sh --input_model=path/to/model \  # model path as *.onnx
                   --config=squeezenet.yaml \
                   --data_path=/path/to/imagenet \
                   --label_path=/path/to/imagenet/label \
                   --output_model=path/to/save

Quantize model with QDQ mode:

bash run_tuning.sh --input_model=path/to/model \  # model path as *.onnx
                   --config=squeezenet_qdq.yaml \
                   --data_path=/path/to/imagenet \
                   --label_path=/path/to/imagenet/label \
                   --output_model=path/to/save

Benchmark

bash run_benchmark.sh --input_model=path/to/model \  # model path as *.onnx
                      --config=squeezenet.yaml \
                      --data_path=/path/to/imagenet \
                      --label_path=/path/to/imagenet/label \
                      --mode=performance # or accuracy