The repository includes MuCAN, LAPAR, Beby-GAN and etc. It is designed for simple training and evaluation.
The training code of BebyGAN and LAPAR is released.
The well-trained models and all visual examples (including BebyGAN and other SOTAs) are available here.
LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-resolution and Beyond
Please find supplementary files of MuCAN and LAPAR here.
-
Clone the repository
git clone https://github.com/Jia-Research-Lab/Simple-SR.git
-
Install the dependencies
- Python >= 3.6
- PyTorch >= 1.2
- spatial-correlation-sampler
pip install spatial-correlation-sampler
- Other packages
pip install -r requirements.txt
-
Download pretrained models from Google Drive. We re-trained the LAPAR models and their results are slightly different from the ones reported in paper.
- MuCAN
- MuCAN_REDS.pth: trained on REDS dataset, 5-frame input, x4 scale
- MuCAN_Vimeo90K.pth: trained on Vimeo90K dataset, 7-frame input, x4 scale
- LAPAR: trained on DIV2K+Flickr2K datasets
Scale x2 Scale x3 Scale x4 LAPAR_A_x2.pth LAPAR_A_x3.pth LAPAR_A_x4.pth LAPAR_B_x2.pth LAPAR_B_x3.pth LAPAR_B_x4.pth LAPAR_C_x2.pth LAPAR_C_x3.pth LAPAR_C_x4.pth - BebyGAN
- RRDB_warmup.pth: provided for initialization
- BebyGAN_x4.pth: trained on DIV2K+Flickr2K datasets, x4 scale
- MuCAN
-
Quick test
You have to define the output_path or gt_path or both. If output_path is given, outputs will be saved. If gt_path is given, PSNR/SSIM will be calculated. If you want to calculate LPIPS results, please install lpips library.
- For SISR,
python3 test_sample.py --sr_type SISR --model_path /model/path --input_path ./demo/SISR/LR_imgs --output_path ./demo/SISR/output --gt_path ./demo/SISR/HR_imgs
- For VSR,
python3 test_sample.py --sr_type VSR --model_path /model/path --input_path ./demo/VSR/LR_imgs --output_path ./demo/VSR/output --gt_path ./demo/VSR/HR_imgs
-
Training Datasets
Download DIV2K and Flickr2K. You may crop the HR and LR images to sub-images for fast reading referring to .utils/data_prep/extract_subimage.py.
-
Evaluation Datasets
Download Set5, Set14, Urban100, BSDS100 and Manga109 from Google Drive uploaded by BasicSR.
-
Update the dataset location in .dataset/__init__.py.
-
(Optional) You can convert images to lmdb files for fast loading referring to BasicSR. And you need to modify the data reading logics in .dataset/*dataset.py accordingly.
-
Create a log folder as
mkdir logs
-
Create a new experiment folder in .exps/. You just need to prepare the config.py and network.py, while the train.py and validate.py are universal. For example, for LAPAR_A_x2, run
cd exps/LAPAR_A_x2/ bash train.sh $GPU_NUM $PORT
Please modify the path of initialization model of BebyGAN (found in Google Drive) in config.py before training.
Notice that you can find the checkpoints, log files and visualization images in either .exps/LAPAR_A_x2/log/ (a soft link) or .logs/LAPAR_A_x2/.
Please refer to validate.py in each experiment folder or quick test above.
We refer to BasicSR for some details.
@inproceedings{li2020mucan,
title={MuCAN: Multi-correspondence Aggregation Network for Video Super-Resolution},
author={Li, Wenbo and Tao, Xin and Guo, Taian and Qi, Lu and Lu, Jiangbo and Jia, Jiaya},
booktitle={European Conference on Computer Vision},
pages={335--351},
year={2020},
organization={Springer}
}
@article{li2020lapar,
title={LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-resolution and Beyond},
author={Li, Wenbo and Zhou, Kun and Qi, Lu and Jiang, Nianjuan and Lu, Jiangbo and Jia, Jiaya},
journal={Advances in Neural Information Processing Systems},
volume={33},
year={2020}
}
@article{li2021best,
title={Best-Buddy GANs for Highly Detailed Image Super-Resolution},
author={Li, Wenbo and Zhou, Kun and Qi, Lu and Lu, Liying and Jiang, Nianjuan and Lu, Jiangbo and Jia, Jiaya},
journal={arXiv preprint arXiv:2103.15295},
year={2021}
}