Few-Shot Domain Adaptation for Low Light RAW Image Enhancement (Project Page)
Enhancing practical low light raw images is a difficult task due to severe noise and color distortions from short exposure time and limited illumination. Despite the success of existing Convolutional Neural Network (CNN) based methods, their performance is not adaptable to different camera domains. In addition, such methods also require large datasets with short-exposure and corresponding long-exposure ground truth raw images for each camera domain, which is tedious to compile. To address this issue, we present a novel few-shot domain adaptation method to utilize the existing source camera labeled data with few labeled samples from the target camera to improve the target domain's enhancement quality in extreme low-light imaging. Our experiments show that only ten or fewer labeled samples from the target camera domain are sufficient to achieve similar or better enhancement performance than training a model with a large labeled target camera dataset. To support research in this direction, we also present a new low-light raw image dataset captured with a Nikon camera, comprising short-exposure and their corresponding long-exposure ground truth images.
Nikon Camera Dataset: Kaggle Link
Environments:
- Python 3.7
- Pytorch 1.0.0 + Rawpy + Numpy + Scipy
- Trained and tested on Tesla V100 32GB GPU. Cuda(>=10.0)
- CPU requirement >= 64 GB
Docker Image: Link
Testing :
- Run "python test.py". We can change "input_dir", "gt_dir", "result_dir" at the beginning.
- We use pack_nikon for Nikon validation and pack_canon for Canon validation.
Model:
- UnetConverter is for the source Grayscale SSIM 16-to-8 bit conversion. The pretrained Sony 16-to-8 bit converter can be downloaded here: Google Drive
- UNetSony is used for shared U-net model (N).