use tensorrt to inference

[Yutong-gannis]

how to use tensorrt to do inference

[qinjian623]

To use TensorRT for inference with your PyTorch model, please follow these steps:

1. Convert your PyTorch model into an ONNX format file. If you haven't done this before, please refer to the PyTorch official documentation: https://pytorch.org/tutorials/advanced/super_resolution_with_onnxruntime.html.
2. Set up your TensorRT environment. The easiest way to do this is to use a Docker container from NVIDIA: https://ngc.nvidia.com/catalog/containers/nvidia:tensorrt.
3. Once your TensorRT environment is set up, you can use the trtexec command to perform a random input inference test and obtain a detailed latency report. Follow the instructions provided by the trtexec command to perform the test.