by Soufiane Belharbi1, Mara KM Whitford2,3, Phuong Hoang2, Shakeeb Murtaza1, Luke McCaffrey2, 3,4, Eric Granger1
1 LIVIA, Dept. of Systems Engineering, ETS Montreal, Canada
2 Goodman Cancer Institute, McGill University, Montreal, Canada
3 Dept. of Biochemistry, McGill University, Montreal, Canada
4 Gerald Bronfman Dept. of Oncology, McGill University, Montreal,
Canada
Confocal fluorescence microscopy is one of the most accessible and widely used imaging techniques for the study of biological processes at the cellular and subcellular levels. Scanning confocal microscopy allows the capture of high-quality images from thick three-dimensional (3D) samples, yet suffers from well-known limitations such as photobleaching and phototoxicity of specimens caused by intense light exposure, which limits its use in some applications, especially for living cells. Cellular damage can be alleviated by changing imaging parameters to reduce light exposure, often at the expense of image quality. Machine/deep learning methods for single-image super-resolution (SISR) can be applied to restore image quality by upscaling lower-resolution (LR) images to produce high-resolution images (HR). These SISR methods have been successfully applied to photo-realistic images due partly to the abundance of publicly available datasets. In contrast, the lack of publicly available data partly limits their application and success in scanning confocal microscopy. In this paper, we introduce a large scanning confocal microscopy dataset named SR-CACO-2 that is comprised of low- and high-resolution image pairs marked for three different fluorescent markers. It allows to evaluate the performance of SISR methods on three different upscaling levels (X2, x34, x8). SR-CACO-2 contains the human epithelial cell line Caco-2 (ATCC HTB-37), and it is composed of 2,200 unique images, captured with four resolutions and three markers, that have been translated in the form of 9,937 patches for experiments with SISR methods. Given the new SR-CACO-2 dataset, we also provide benchmarking results for 16 state-of-the-art methods that are representative of the main SISR families. Results show that these methods have limited success in producing high-resolution textures, indicating that SR-CACO-2 represents a challenging problem. The dataset is released under a Creative Commons license (CC BY-NC-SA 4.0), and it can be accessed freely. Our dataset, code and pretrained weights for SISR methods are publicly available : https://github.com/sbelharbi/sr-caco-2.
Code: Pytorch 2.0.0
@inproceedings{belharbi24-sr-caco-2,
title={SR-CACO-2: A Dataset for Confocal Fluorescence Microscopy Image Super-Resolution},
author={Belharbi, S. and Whitford, M.K.M. and Hoang, P. and Murtaza, S. and McCaffrey, L. and Granger, E.},
booktitle={NeurIPS},
year={2024}
}
- SR-CACO-2 dataset: LICENSE/download
- Install
- Dataset path
- Run code
- Pretrained weights (evaluation)
- SR-CACO-2 presentation
THIS SR-CACO-2 DATASET IS LICENSED UNDER CREATIVE COMMONS ATTRIBUTION-NONCOMMERCIAL-SHAREALIKE 4.0 INTERNATIONAL LICENSE (CC BY-NC-SA 4.0). HTTPS://CREATIVECOMMONS.ORG/LICENSES/BY-NC-SA/4.0/
# Create a virtual env.
./create_env.sh NAME_OF_THE_VIRTUAL_ENVOnce you download the dataset, you need to adjust the paths in get_root_datasets().
Run code :
- E.g.: Train
SwinIRmethod,CELL2,X8:
#!/usr/bin/env bash
# Activate your virtual env.
# ==============================================================================
cudaid=$1
export CUDA_VISIBLE_DEVICES=$cudaid
export OMP_NUM_THREADS=50
python main.py \
--task super-resolution \
--scale 8 \
--method SWINIR \
--net_type swinir \
--train_dsets caco2_train_X_8_in_64_out_512_cell_CELL2 \
--valid_dsets caco2_val_X_8_in_64_out_512_cell_CELL2 \
--test_dsets caco2_test_X_8_in_64_out_512_cell_CELL2 \
--n_channels 1 \
--valid_n_samples 128 \
--checkpoint_eval 0.9 \
--checkpoint_save 0.9 \
--test_epoch_freq 10000 \
--use_interpolated_low False \
--inter_low_th 7.0 \
--inter_low_sigma 6.0 \
--dist_backend gloo \
--distributed False \
--eval_bsize 1 \
--cudaid $cudaid \
--debug_subfolder SURVEY \
--amp False \
--sample_tr_patch roi \
--h_size 128 \
--train_n 1.0 \
--sample_tr_patch_th_style fix_threshold \
--sample_tr_patch_th 4.0 \
--eval_over_roi_also True \
--eval_over_roi_also_model_select True \
--da_blur False \
--da_dot_bin_noise False \
--da_add_gaus_noise False \
--G_optimizer_type sgd \
--max_epochs 100 \
--G_scheduler_type MyStepLR \
--batch_size 8 \
--G_scheduler_gamma 0.5 \
--G_optimizer_wd 0.0 \
--G_scheduler_step_size 30 \
--G_optimizer_lr 0.01 \
--w_sparsity False \
--swinir_window_size 8 \
--swinir_depths 6+6+6+6 \
--swinir_embed_dim 180 \
--swinir_num_heads 6+6+6+6 \
--swinir_mlp_ratio 2 \
--swinir_upsampler pixelshuffledirect \
--ppiw False \
--l1 False \
--l2 True \
--l2_use_residuals False \
--l2_lambda 1.0 \
--l2sum False \
--l2sum_use_residuals False \
--l2sum_lambda 1.0 \
--ssim True \
--ssim_lambda 5.0 \
--ssim_window_s 19 \
--charbonnier False \
--boundpred False \
--local_moments False \
--img_grad False \
--norm_img_grad False \
--laplace False \
--norm_laplace False \
--loc_var False \
--norm_loc_var False \
--norm_loc_var_ksz 3 \
--norm_loc_var_use_residuals False \
--norm_loc_var_lambda 1.0 \
--norm_loc_var_type 2 \
--hist False \
--kde False \
--ce False \
--elb_init_t 1.0 \
--elb_max_t 10.0 \
--elb_mulcoef 1.01 \
--exp_id 01_14_2024_00_11_43_277115__2942919We provide the weights for all the models (144 models: 16 methods x 3 cells
x 3 scales). Weights can be found at Hugging Face in the file shared-trained-models.tar.gz.
To run a single case, e.g. for ACT method, CELL0, X2:
python eval.py --cudaid 0 --exp_path $root/shared-trained-models/SURVEY_ABLATIONS/super-resolution/ACT/caco2_train_X_2_in_256_out_512_cell_CELL0/id_12_21_2023_07_59_15_641499__1617383-tsk_super-resolution-x_2-netG_ACT-sd_0-l2_yes-ssim_yesTo run all 144 cases:
./eval_all.sh 0The provided weights can be used to reproduce the reported results in the paper in the paper:
The file share-visualization-30-samples-test.zip contains visual predictions on the test set.
