ONNX model, dependency to pytorch

[joihn]

Hi,
Thanks for this work
Once the model is converted to ONNX/TensorRT (including the NMS), there is no more dependency to pytorch, right ?

[zhiqwang]

Hi @joihn , it depends on your needs here, the C++ interface https://github.com/zhiqwang/yolov5-rt-stack/tree/main/deployment/tensorrt doesn't rely on PyTorch, but we use some functions of PyTorch to implement the pre-processing part of the Python inference interface
https://github.com/zhiqwang/yolov5-rt-stack/blob/0c88e4f44646092078d5d55caed575dc2d26823d/yolort/runtime/y_tensorrt.py#L193-L194

So you can totally remove the PyTorch dependencies if you use the OpenCV and pycuda to do the pre-processing. In other words, it actually means that you need to rewrite the pre-processing of PredictorTRT with OpenCV and pycuda, which is a relatively minor effort. Check TensorRT samples here https://github.com/NVIDIA/TensorRT/blob/f4a8635/samples/python/detectron2/infer.py#L114 for more details. And feel free to pin me if you have any problems here.

https://github.com/zhiqwang/yolov5-rt-stack/blob/0c88e4f44646092078d5d55caed575dc2d26823d/yolort/runtime/y_tensorrt.py#L159-L160