-
Notifications
You must be signed in to change notification settings - Fork 606
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mmdet TensorRT support #1042
base: main
Are you sure you want to change the base?
Mmdet TensorRT support #1042
Conversation
@enesozi, amazing contribution! Can you please fix the tests? |
@fcakyon mmdeploy might be annoying when it comes to installing it. I hope the latest changes fix it. |
Weirdly, the workflow is trying to run the command mim install mmdeploy==1.3.1 twice. So, the command looks like this mim install mmdeploy==1.3.1 mim - another library called mim. |
I checked the mmdet_tests locally and confirmed they work. CI pipeline now fails at huggingface tests. ERROR: test_perform_inference (tests.test_huggingfacemodel.TestHuggingfaceDetectionModel)Traceback (most recent call last): |
.github/workflows/ci.yml
Outdated
@@ -85,8 +85,7 @@ jobs: | |||
run: pip install torch==1.13.1 torchvision==0.14.1 | |||
|
|||
- name: Install MMDetection(3.0.0) with MMCV(2.0.0) | |||
run: > |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mmdeploy needs openmim installed
GPU Info: NVIDIA GeForce RTX 3090
Model used: rtmdet_l_8xb32-300e_coco.py
Deployment cfg file used: detection_tensorrt-fp16_static-640x640.py -> you may find examples here
Average inference times(Just the _call_ method of the wrappers used):
Average inference times(get_sliced_prediction function):
The difference was around 400 ms but it might yield significant speed improvement for edge devices such as jetson.
Disclaimer: This implementation is not perfect as it could be generalized to work for other frameworks(yolo, detectron2, etc.). I needed this for my current project and wanted to share it as a base for anyone interested.
Example usage: