Computer Vision Lab, CAIDAS, University of Würzburg
- AIM 2022 Reversed ISP Challenge Official repo!
- Model-Based Image Signal Processors via Learnable Dictionaries AAAI 2022 Oral - Official repo!
- MAI 2022 Learned ISP Challenge Complete Baseline solution
- Citation and Acknowledgement
- Contact
In this challenge, we look for solutions to recover RAW readings from the camera using only the corresponding RGB images processed by the in-camera ISP. Successful solutions should generate plausible RAW images, and by doing this, other downstream tasks like Denoising, Super-resolution or Colour Constancy can benefit from such synthetic data generation. Click here to read more information about the challenge.
- aim-starter-code.ipynb - Simple dataloading and visualization of RGB-RAW pairs + other utils.
- aim-baseline.ipynb - End-to-end guide to load the data, train a simple UNet model and make your first submission!
Model-Based Image Signal Processors via Learnable Dictionaries (AAAI '22 Oral)
Project website where you can find the poster, presentation and more information.
Hybrid model-based and data-driven approach for modelling ISPs using learnable dictionaries. We explore RAW image reconstruction and improve downstream tasks like RAW Image Denoising via raw data augmentation-synthesis.
The code will be released soon. If you have implementation questions or you need qualitative samples for comparison, please contact me.
We provide the figure/illustration of our method in mbispld.
You can find at mai22-learnedisp and end-to-end baseline: dataloading, training top solution, model conversion to tflite. The model achieved 23.46dB PSNR after training for a few hours. Here you can see a sample RAW input and the resultant RGB.
We test the model on AI Benchmark. The model average latency is 60ms using a input RAW image 544,960,4 and generating a RGB 1088,1920,3, in a mid-level smartphone (45.4 AI-score) using Delegate GPU and FP16.
[1] Model-Based Image Signal Processors via Learnable Dictionaries by Conde et al, AAAI 2022.
[2] Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report by Ignatov et al, CVPRW 2021.
[3] Learning Raw Image Reconstruction-Aware Deep Image Compressors by Abhijith Punnappurath and Michael S. Brown, TPAMI 2019.
[4] Unprocessing Images for Learned Raw Denoising by Brooks et al. , CVPR 2019
[5] CycleISP: Real Image Restoration via Improved Data Synthesis by Zamir et al. , CVPR 2020
Related Challenges
Mobile AI & AIM 2022 Learned Smartphone ISP Challenge organized by Andrey Ignatov.
@inproceedings{conde2022model,
title={Model-Based Image Signal Processors via Learnable Dictionaries},
author={Conde, Marcos V and McDonagh, Steven and Maggioni, Matteo and Leonardis, Ales and P{\'e}rez-Pellitero, Eduardo},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
volume={36},
number={1},
pages={481--489},
year={2022}
}
@inproceedings{conde2022aim,
title={{R}eversed {I}mage {S}ignal {P}rocessing and {RAW} {R}econstruction. {AIM} 2022 {C}hallenge {R}eport},
author={Conde, Marcos V and Timofte, Radu and others},
booktitle={Proceedings of the European Conference on Computer Vision Workshops (ECCVW)},
year={2022}
}
Marcos Conde (marcos.conde-osorio@uni-wuerzburg.de) and Radu Timofte (radu.timofte@uni-wuerzburg.de) are the contact persons and direct managers of the AIM challenge. Please add in the email subject "AIM22 Reverse ISP Challenge" or "AISP"