-
Notifications
You must be signed in to change notification settings - Fork 648
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] try to output probability map #2050
Comments
@yuanyuangoo hi, the channel number of preprocessed image is 8, which should be 3. Could you post here your deploy config and pipeline.json? |
pipeline.json is |
deploy.json |
@yuanyuangoo hi, could you try again the mmdeploy with commit id after this PR #2038? |
Hi RunningLeon, I am on branch main and my branch is up to date with 'origin/main'. |
@yuanyuangoo hi, sorry for the trouble. You could add example pipelin.json
|
Thank you @RunningLeon, It works. Could it be possible that the pipeline.json generated by mmdeploy/tools/deploy.py includes "with_argmax": false automatically, when I have specify "codebase_config = dict(with_argmax=False)" in my mmdeploy/configs/mmseg/segmentation_tensorrt-fp16_static-1024x1224.py file? So that I don't have to manually add "with_argmax": false to pipeline.json everytime I deploy model. |
Another issue is that when I output the possibility map following your instruction, the processing speed is about 8 times slower than just outputting the segmentation results. Maybe It's caused by the fact that the possibility map is larger than the segmentation map, so it takes a longer time to transfer from GPU to the local machine. Is there any way to solve this issue, please? |
Additionally, to get the actual confidence of every class, I still need to add a softmax after the output. Could it be possible to support this in the tensorrt engine? |
|
The script I use to test speed is as
On my 3080 graphic card under tensorrt-fp16 model, I got about 7 fps with_argmax=False, and 50 fps with_argmax=True. |
Can you give me a hint on how to achieve that, please? |
I tried to replace in file mmdeploy/codebase/mmseg/models/segmentors/encoder_decoder.py:30
to
and now I can get all confidence right. But the processing speed is still like 10 times lower than just outputting the class maps. Is there any way to solve this, please? |
@yuanyuangoo, hi, you are right for outputing scores. Because you add a softmax layer, the fps is slower. As for why it's 10 times slower, you have to profiler by yourself and determine the proportion of each layer to the total consuming time. |
@RunningLeon I just tried to tensorrt python api to load the engine file and run the inference, and I can get more than 20 fps, including all the preprocessing and post processing . But when I use mmdeploy python runtime, I can only have like 3-5 fps. From the comparison, it's clear that mmdeploy has potential to be much faster. |
@yuanyuangoo, Comparision between two backends needs careful design. How did you test with TensorRT python api? Pls. make sure the pre-and post-processing are included. BTW, from personal experience, |
|
The above is the inference script. I modified code in /root/workspace/mmdeploy/mmdeploy/codebase/mmseg/models/segmentors/encoder_decoder.py, replace "seg_logit" in line 31 to "seg_logit.softmax(dim=1).max(dim=1, keepdim=True)" I also added "codebase_config = dict(with_argmax=False)" to the end of "/root/workspace/mmdeploy/configs/mmseg/segmentation_tensorrt-fp16_static-1024x1224.py" After this I converted pytorch model to trt engine, ran inference, and I got like 30+ fps. |
This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response. |
This issue is closed because it has been stale for 5 days. Please open a new issue if you have similar issues or you have any new updates now. |
Checklist
Describe the bug
I try to convert a mmseg model to tensorrt engine to output probability map by put codebase_config = dict(with_argmax=False) in th config file. As a result, it can output probability map. But I got this.
[2023-05-05 14:21:21.583] [mmdeploy] [error] [segment.cpp:45] probability feat map with shape: [1, 8, 1024, 1224] requires
with_argmax_=false
Reproduction
import time
import numpy as np
from mmdeploy_python import Segmentor
from mmdeploy_runtime import Segmentor
segmentor = Segmentor(
model_path='./deeplabv3plus_bud/', device_name='cuda', device_id=0)
prev_frame_time = 0
new_frame_time = 0
im = np.zeros((1224, 1024, 3))
seg = segmentor(im)
Environment
Error traceback
[2023-05-05 14:21:21.583] [mmdeploy] [error] [segment.cpp:45] probability feat map with shape: [1, 8, 1024, 1224] requires `with_argmax_=false`
The text was updated successfully, but these errors were encountered: