How to visualize the sampling positions?

[pumpkinnan97]

Hello, your work is so great!
I am interested in your implementation details about the visualization of the sampling position in Fig.5 and Fig.6 of your paper. Could you please tell me more about the implementation details about it? I can't find the code in this repo.
Thanks!

[YapengTian]

Thanks for your interest in our work. Previously, I printed the predict offsets and visualized the receptive fields accordingly. Read the model.py L2O2-206, we can know that sampling positions are decided by the 3x3 kernels and two predicted offsets which are used to sample support frames.

For one pixel in the predicted LR frame, it is corresponding to 9 pixels in aligned_fea. Then, for a pixel in aligned_fea, it is corresponding to 9 pixels (actual number should be 9^8 since we use the dconv grounp=8. In practice, a lot of pixels might be the same and we just use the most far away pixels for each axis) in fea. Furthermore, we can find associated positions in supp. In this way, we can visualize all 9^3 positions. But note the actual receptive field with propagated to image-level should be even larger since we have feature extraction module before alignment.

[xiaomingxige]

I want to run on windows!

[pumpkinnan97]

Thanks for your interest in our work. Previously, I printed the predict offsets and visualized the receptive fields accordingly. Read the model.py L2O2-206, we can know that sampling positions are decided by the 3x3 kernels and two predicted offsets which are used to sample support frames.

For one pixel in the predicted LR frame, it is corresponding to 9 pixels in aligned_fea. Then, for a pixel in aligned_fea, it is corresponding to 9 pixels (actual number should be 9^8 since we use the dconv grounp=8. In practice, a lot of pixels might be the same and we just use the most far away pixels for each axis) in fea. Furthermore, we can find associated positions in supp. In this way, we can visualize all 9^3 positions. But note the actual receptive field with propagated to image-level should be even larger since we have feature extraction module before alignment.

Thanks for your reply! Now I know that you print the dconv offsets in the image. I understand how to implement it in theory. But I still can't understand how to implement it in code. If that's convenient, can you show me the code of this part?
I'm really very interested in your work, and your paper looks great so that I want to obtain more details, thanks again for your code in this repo and helping!

[sunnyHelen]

Where did you see the visualization of the sampling position? I also want to see that. Where can I get the supplementary? @YapengTian @pumpkinnan97

[YapengTian]

Please check the CVPR VERSION: https://openaccess.thecvf.com/content_CVPR_2020/papers/Tian_TDAN_Temporally-Deformable_Alignment_Network_for_Video_Super-Resolution_CVPR_2020_paper.pdf