Suggestions For Improvement of Feature Visualization

[Zengyf-CVer]

@glenn-jocher
First of all, thank you for your repair in Feature visualization. I have some new improvement ideas for Feature visualization:
I drew my improvement idea into a picture:

step:

• First find the feature map belonging to the output image in Stage23_C3_features through the output image. (Select 1 from 512)
• Then at each stage, the corresponding feature map is selected according to this idea.
• Finally, the 24 stages output 24 corresponding feature maps.

Since my level is far from enough, so please help me realize it. I think this idea is very practical, thank you.

[glenn-jocher]

@Zengyf-CVer hey thanks for the detailed graphics! I think I understand what you are saying but I also think you may be misunderstanding the flow of information through the model.

The various stages have various channel counts, but you could for example plot channel 0 through all the stages, i.e. 23 plots, but I doubt that the plots would have much to do with each other. For example in a 32x64 convolution all 32 input channels are convolved with all 64 output channels, so channel 0 input and channel 0 output are not related anymore than channel 0 input to channel 63 output.

That's my understanding at least. So you'd essentially be looking at 23 unrelated channel 0 feature maps.

Perhaps the main benefit would be in seeing the shape/resolution of the feature maps change through the model.

[glenn-jocher]

@Zengyf-CVer we can definitely do this of course, the main challenge is implementing this in such a way as to minimally impact the forward method so that we are not hurting the much wider use-case of normal inference.

[Zengyf-CVer]

@glenn-jocher
Thank you for your detailed response to my suggestion. Through your explanation, I have gained a lot.
As you said, the feature maps of each stage are weakly correlated, and the 23 feature maps obtained may be uncorrelated.My main idea is to select feature maps that are similar to the output image at each stage, even if there is a weak relationship in each stage, it will have no effect.
In this way, we can perform a more intuitive analysis on the feature maps of several stages close to the output layer, so as to further describe the interpretability of the model to the object detection process, and lay the foundation for the improvement of the function and performance of the neural network.

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!