Awesome-Inpainting-Tech

A curated list of inpainting papers and resources, inspired by awesome-computer-vision.

Contents

• Image Inpainting
• Video Inpainting
• Challenge

Image Inpainting

Classical methods (Non-learning based)

1. Image inpainting. Bertalmio, M., Sapiro, G., Caselles, V., & Ballester, C. In SIGGRAPH 2000
2. Simultaneous structure and texture image inpainting. Bertalmio, M., Vese, L., Sapiro, G., & Osher, S. In TIP 2003
3. Region filling and object removal by exemplar-based image inpainting. Criminisi, A., Pérez, P., & Toyama, K. In TIP 2004
4. Image completion with structure propagation. Sun, J., Yuan, L., Jia, J., & Shum, H. Y. In TOG 2005
5. Image completion using planar structure guidance. Huang, J. B., Kang, S. B., Ahuja, N., & Kopf, J. In TOG 2014 [code] [project]
6. PatchMatch: A randomized correspondence algorithm for structural image editing. Connelly Barnes, Eli Shechtman, Adam Finkelstein, Dan B Goldman. In TOG 2009, [project]
7. Deep Image Prior. Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky. In IJCV 2020, [project] [code] [demo]

Deep Architectures (Learning Based)

1. Shepard convolutional neural networks. Ren, J. S., Xu, L., Yan, Q., & Sun, W. In NeurIPS 2015 [code]
2. Context encoders: Feature learning by inpainting. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. In CVPR 2016. [code]
3. Globally and locally consistent image completion. Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017). In TOG 2017 [code] [project]
4. High-resolution image inpainting using multi-scale neural patch synthesis. Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., & Li, H. In CVPR 2017 [code]
5. Generative face completion. Li, Y., Liu, S., Yang, J., & Yang, M. H. In CVPR 2017 [code]
6. Semantic image inpainting with deep generative models. Yeh, R. A., Chen, C., Yian Lim, T., Schwing, A. G., Hasegawa-Johnson, M., & Do, M. N. In CVPR 2017 [code] [project]
7. Generative image inpainting with contextual attention. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. In CVPR 2018 [code] [project]
8. Natural and effective obfuscation by head inpainting. Sun Qianru et al. In CVPR 2018
9. Eye in-painting with exemplar generative adversarial networks. Dolhansky, B., & Canton Ferrer, C. In CVPR 2018 [project] [code]
10. Uv-gan: Adversarial facial uv map completion for pose-invariant face recognition. Deng, J., Cheng, S., Xue, N., Zhou, Y., & Zafeiriou, S. In CVPR 2018
11. Disentangling Structure and Aesthetics for Style-aware Image Completion. Gilbert, A., Collomosse, J., Jin, H., & Price, B. In CVPR 2018
12. Image inpainting for irregular holes using partial convolutions. Liu, G., Reda, F. A., Shih, K. J., Wang, T. C., Tao, A., & Catanzaro, B. In ECCV 2018 [project]
13. Contextual-based image inpainting: Infer, match, and translate. Song, Y., Yang, C., Lin, Z., Liu, X., Huang, Q., Li, H., & Jay Kuo, C. C. In ECCV 2018
14. Shift-net: Image inpainting via deep feature rearrangement. Yan, Z., Li, X., Li, M., Zuo, W., & Shan, S. In ECCV 2018 [code]
15. Image Inpainting via Generative Multi-column Convolutional Neural Networks. Wang, Y., Tao, X., Qi, X., Shen, X., & Jia, J. In NeurIPS 2018 [code]
16. SPG-Net: Segmentation prediction and guidance network for image inpainting. Song, Y., Yang, C., Shen, Y., Wang, P., Huang, Q., & Kuo, C. C. J. In BMVC 2018
17. Structural inpainting. Vo, H. V., Duong, N. Q., & Pérez, P. In MM 2018
18. Semantic Image Inpainting with Progressive Generative Networks. Zhang, H., Hu, Z., Luo, C., Zuo, W., & Wang, M. In MM 2018 [code]
19. Face Completion with Semantic Knowledge and Collaborative Adversarial Learning. Liao, H., Funka-Lea, G., Zheng, Y., Luo, J., & Zhou, S. K. In ACCV 2018
20. Edge-Aware Context Encoder for Image Inpainting. Liao, L., Hu, R., Xiao, J., & Wang, Z. In ICASPP 2018
21. Faceshop: Deep sketch-based face image editing. Portenier, T., Hu, Q., Szabó, A., Bigdeli, S. A., Favaro, P., & Zwicker, M. In TOG 2018
22. High resolution face completion with multiple controllable attributes via fully end-to-end progressive generative adversarial networks. Chen, Z., Nie, S., Wu, T., & Healey, C. G. In Arxiv 2018
23. On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs. Banerjee, S., Scheirer, W. J., Bowyer, K. W., & Flynn, P. J. In Arxiv 2018
24. Pluralistic Image Completion. Zheng, C., Cham, T. J., & Cai, J. In CVPR 2019 [code] [project]
25. Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting. Zeng et al., In CVPR 2019 [code]
26. Foreground-aware Image Inpainting. Xiong et al., In CVPR 2019
27. Deep Reinforcement Learning of Volume-guided Progressive View Inpainting for 3D Point Scene Completion from a Single Depth Image. Han, X., Zhang, Z., Du, D., Yang, M., Yu, J., Pan, P., ... & Cui, S. In CVPR 2019
28. PEPSI: Fast Image Inpainting With Parallel Decoding Network. CVPR (pp. 11360-11368). Sagong et al. In CVPR 2019
29. Coordinate-Based Texture Inpainting for Pose-Guided Human Image Generation. Grigorev et al. CVPR2019.
30. Deep Inception Generative Network for Cognitive Image Inpainting. Xiao, Q., Li, G., & Chen, Q. In Arxiv 2019
31. Detecting Overfitting of Deep Generative Networks via Latent Recovery. Webster, R., Rabin, J., Simon, L., & Jurie, F. In Arxiv 2019
32. SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color. Jo, Y., & Park, J. (2019). In Arxiv 2019 [code]
33. Deep Fusion Network for Image Completion. Hong et al. In Arxiv 2019 [code]
34. PEPSI++: Fast and Lightweight Network for Image Inpainting. Shin et al. In Arxiv 2019
35. Generative Image Inpainting with Submanifold Alignment Li et al. In IJCAI 2019
36. MUSICAL: Multi-Scale Image Contextual Attention Learning for Inpainting. Wang, N., Li, J., Zhang, L., & Du, B. In IJCAI 2019
37. Coarse-to-Fine Image Inpainting via Region-wise Convolutions and Non-Local Correlation. Ma, Y., Liu, X., Bai, S., Wang, L., He, D., & Liu, A. In IJCAI 2019
38. StructureFlow: Image Inpainting via Structure-aware Appearance Flow, Yurui Ren, Xiaoming Yu, Ruonan Zhang, Thomas H. Li, Shan Liu, Ge Li. In ICCV 2019 [code]
39. Image Inpainting with Learnable Bidirectional Attention Maps, Xie et al. In ICCV 2019 [code]
40. Coherent Semantic Attention for Image Inpainting, Liu et al. In ICCV 2019 [code]
41. EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning. Nazeri, K., Ng, E., Joseph, T., Qureshi, F., & Ebrahimi, M. In ICCVW 2019 [code]
42. Free-form image inpainting with gated convolution. Yu et al. In ICCV 2019 [project]
43. Region Normalization for Image Inpainting. Tao et al. In AAAI 2020 [code]
44. Learning to Incorporate Structure Knowledge for Image Inpainting, Yang et al. In AAAI 2020 [code]
45. Prior Guided GAN Based Semantic Inpainting, Lahiri1 et al. In CVPR 2020
46. UCTGAN: Diverse Image Inpainting based on Unsupervised Cross-Space Translation, Zhao et al. In CVPR 2020
47. Recurrent Feature Reasoning for Image Inpainting, Li et al. In CVPR 2020 [code]
48. Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting, Yi et al., In CVPR 2020 [code]
49. 3D Photography using Context-aware Layered Depth Inpainting, Shih et al. In CVPR 2020 [demo] [code]
50. Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations, Liu et al. In ECCV 2020 [code]
51. Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes, Liao et al. In ECCV 2020
52. VCNet: A Robust Approach to Blind Image Inpainting, Wang et al. In ECCV2020 [code]
53. High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling, Zeng et al. In ECCV 2020 [project]
54. Image Inpainting with Onion Convolution, Shant et al., In ACCV2020
55. Hyperrealistic Image Inpainting with Hypergraphs, Wadhwa et al., In WACV 2021 [code]
56. Image Inpainting with External-internal Learning and Monochromic Bottleneck, Wang et al., In CVPR 2021 [code] [project]
57. Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE, Peng et al., In CVPR 2021 [code]
58. PD-GAN: Probabilistic Diverse GAN for Image Inpainting, Liu et al., In CVPR 2021 [code]
59. Image Inpainting Guided by Coherence Priors of Semantics and Textures, In CVPR 2021
60. TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations, Zhou et al., In CVPR 2021 [code] [project]
61. FaceInpainter: High Fidelity Face Adaptation to Heterogeneous Domains, Li et al., In CVPR 2021
62. Large Scale Image Completion via Co-Modulated Generative Adversarial Networks, Zhao et al., In ICLR 2021 [code]
63. Image Inpainting via Conditional Texture and Structure Dual Generation, Guo et al., In ICCV 2021 [code]
64. CR-Fill: Generative Image Inpainting With Auxiliary Contextual Reconstruction, Zeng et al., In ICCV 2021 [code]
65. Parallel Multi-Resolution Fusion Network for Image Inpainting, Wang et al., In ICCV 2021
66. WaveFill: A Wavelet-Based Generation Network for Image Inpainting, Yu et al., In ICCV 2021
67. SLIDE: Single Image 3D Photography With Soft Layering and Depth-Aware Inpainting, Kaeser et al., In ICCV 2021 [project]
68. Distillation-Guided Image Inpainting, Suin et al., In ICCV 2021
69. High-Fidelity Pluralistic Image Completion With Transformers, Wan et al., In ICCV 2021 [code] [project]
70. Learning a Sketch Tensor Space for Image Inpainting of Man-Made Scenes, Cao et al., In ICCV 2021 [project] [code]
71. Learning High-Fidelity Face Texture Completion Without Complete Face Texture, Kim et al., In ICCV 2021

Video Inpainting

Classical methods (Non-learning based)

1. Navier-stokes, fluid dynamics, and image and video inpainting. Bertalmio, M., Bertozzi, A. L., & Sapiro, G. In CVPR 2001 (Vol. 1, pp. I-I). IEEE.
2. Video inpainting of occluding and occluded objects. Patwardhan, K. A., Sapiro, G., & Bertalmio, M. In ICIP 2005 (Vol. 2, pp. II-69). IEEE.
3. Full-frame video stabilization with motion inpainting. Matsushita, Y., Ofek, E., Ge, W., Tang, X., & Shum, H. Y. In TPAMI 2006, (7), 1150-1163. [project]
4. Video completion by motion field transfer. Shiratori, T., Matsushita, Y., Tang, X., & Kang, S. B. (2006, June). In CVPR 2006 (Vol. 1, pp. 411-418). [project]
5. Space-time completion of video. Wexler, Y., Shechtman, E., & Irani, M. In TPAMI 2007, (3), 463-476. [project]
6. Video inpainting under constrained camera motion. Patwardhan, K. A., Sapiro, G., & Bertalmío, M. In TIP 2007, 16(2), 545-553.
7. How not to be seen—object removal from videos of crowded scenes. Granados, M., Tompkin, J., Kim, K., Grau, O., Kautz, J., & Theobalt, C. In Computer Graphics Forum 2012 (Vol. 31, No. 2pt1, pp. 219-228). [project]
8. Background inpainting for videos with dynamic objects and a free-moving camera. Springer, Berlin, Heidelberg. Granados, M., Kim, K. I., Tompkin, J., Kautz, J., & Theobalt, C. In ECCV 2012. [project]
9. Video inpainting of complex scenes. Newson, A., Almansa, A., Fradet, M., Gousseau, Y., & Pérez, P. In SIAM Journal on Imaging Sciences 2014, 7(4), 1993-2019. [project]
10. Video inpainting with short-term windows: application to object removal and error concealment. Ebdelli, M., Le Meur, O., & Guillemot, C. In TIP 2015, 24(10), 3034-3047.
11. Temporally coherent completion of dynamic video. Huang, J. B., Kang, S. B., Ahuja, N., & Kopf, J. In TOG 2016. [project] [code]

Deep Architectures (Learning Based)

1. Video inpainting by jointly learning temporal structure and spatial details. Wang, C., Huang, H., Han, X., & Wang, J. In AAAI 2019
2. Deep Flow-Guided Video Inpainting. Xu, R., Li, X., Zhou, B., & Loy, C. C. In CVPR 2019 [code] [project]
3. Deep Video Inpainting. Kim, D., Woo, S., Lee, J. Y., & Kweon, I. S. In CVPR 2019 [code] [project]
4. Deep Blind Video Decaptioning by Temporal Aggregation and Recurrence. Kim, D., Woo, S., Lee, J. Y., & Kweon, I. S. In CVPR 2019 [project]
5. VORNet: Spatio-temporally Consistent Video Inpainting for Object Removal Ya-Liang Chang, Zhe Yu Liu, Winston Hsu. In CVPRW 2019 [code]
6. Learnable Gated Temporal Shift Module for Deep Video Inpainting Ya-Liang Chang, Zhe Yu Liu, Kuan-Ying Lee, Winston Hsu. In BMVC 2019 [code]
7. Align-and-Attend Network for Globally and Locally Coherent Video Inpainting. Woo, S., Kim, D., Park, K., Lee, J. Y., & Kweon, I. S. In Arxiv 2019
8. Frame-Recurrent Video Inpainting by Robust Optical Flow Inference Yifan Ding, Chuan Wang, Haibin Huang, Jiaming Liu, Jue Wang, Liqiang Wang. In Arxiv 2019
9. Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN Chang et al., In ICCV 2019 [code]
10. Onion-Peel Networks for Deep Video Completion, Oh et al., In ICCV 2019 [project] [code]
11. Copy-and-Paste Networks for Deep Video Inpainting, Lee et al., In ICCV 2019 [code][project]
12. An Internal Learning Approach to Video Inpainting, Zhang et al., In ICCV 2019 [project]
13. Short-Term and Long-Term Context Aggregation Network for Video Inpainting, Li et al. In ECCV 2020
14. Proposal-based Video Completion, Hu et al. In ECCV 2020
15. DVI: Depth Guided Video Inpainting for Autonomous Driving, Liao et al. In ECCV 2020 [project]
16. Learning Joint Spatial-Temporal Transformations for Video Inpainting, Zeng et al., In ECCV 2020 [project]
17. Flow-edge Guided Video Completion, Gao et al. In ECCV 2020 [project]
18. Progressive Temporal Feature Alignment Network for Video Inpainting, Zou et al., In CVPR 2021 [code]
19. Internal Video Inpainting by Implicit Long-range Propagation, ouyang et al., In ICCV 2021 [project] [code]
20. FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting, Liu et al., In ICCV 2021 [code]
21. Flow-Guided Video Inpainting With Scene Templates, Lao et al., ```In ICCV 2021''' [code]
22. Frequency-Aware Spatiotemporal Transformers for Video Inpainting Detection, Yu et al., ```In ICCV 2021'''
23. Occlusion-Aware Video Object Inpainting, ke et al., ```In ICCV 2021''' [project]

Challenge

1. Looking at People ECCV Satellite Challenge @ ECCV 2018
2. ICME Grand Challenge Learning-Based Image Inpainting @ ICME 2019
3. Image Inpainting Challenge - AIM Workshop and Challenges @ ECCV 2020
4. Video Inpainting Challenge - MangoTV International Video Inpainting Challenge @ 2021