🔥 🔥 🔥 [22/12/2023] The pre-trained model and the code for real-world inference, training and testing of our follow-up work are now available
Note
If you are interested in analog video restoration take a look at our follow-up work "Reference-based Restoration of Digitized Analog Videotapes" accepted at WACV 2024.
This application is part of the ReInHerit Toolkit.
This is the official repository of "Restoration of Analog Videos Using Swin-UNet" [Demo ACM MM 2022].
In this work, we present an approach to restore analog videos of historical archives. These videos often contain severe visual degradation due to the deterioration of their tape supports that require costly and slow manual interventions to recover the original content. The proposed method uses a multi-frame approach and is able to deal also with severe tape mistracking, which results in completely scrambled frames. Tests on real-world videos from a major historical video archive show the effectiveness of our approach.
To get a local copy up and running follow these simple steps.
We strongly recommend the use of the Anaconda package manager in order to avoid dependency/reproducibility problems. A conda installation guide for Linux systems can be found here.
- Clone the repo
git clone https://github.com/miccunifi/analog-video-restoration.git- Install Python dependencies
conda create -n analog_video_restoration -y python=3.9
conda activate analog_video_restoration
pip install -r requirements.txtSee the repo of our follow-up work to download the synthetic dataset.
- Make your training dataset have the following structure:
<dataset-name>
└─── train
└─── input
└─── 000
| 00000.jpg
| 00001.jpg
| ...
└─── 001
| 00000.jpg
| 00001.jpg
| ...
...
└─── gt
└─── 000
| 00000.jpg
| 00001.jpg
| 00002.jpg
| ...
└─── 001
| 00000.jpg
| 00001.jpg
| ...
...
└─── val
└─── input
└─── 000
| 00000.jpg
| 00001.jpg
| ...
└─── 001
| 00000.jpg
| 00001.jpg
| ...
...
└─── gt
└─── 000
| 00000.jpg
| 00001.jpg
| 00002.jpg
| ...
└─── 001
| 00000.jpg
| 00001.jpg
| ...
...
-
Get your Comet api key for online logging of the losses and metrics
-
Run the training code with
python src/train.py --experiment-name video_swin_unet --data-base-path <path-to-dataset> --devices 0 --api-key <your-Comet-api-key> --batch-size 2 --num-epochs 100 --num-workers 20 --pixel-loss-weight 200 --perceptual-loss-weight 1
-
If needed, download the pretrained model from Google Drive and copy it inside the folder
pretrained_models/video_swin_unet/ -
Extract the frames of the video in .jpg images and save them in a folder
mkdir <folder-name>
ffmpeg -i <video-file-name> -qscale:v 2 <folder-name>/%00d.jpg
- Run inference on the folder with
python src/real_world_test.py --experiment-name video_swin_unet --data-base-path <path-to-folder> --results-path results --patch-size 512 --fps 60
If you find this work useful for your research, please consider citing:
@inproceedings{10.1145/3503161.3547730,
author = {Agnolucci, Lorenzo and Galteri, Leonardo and Bertini, Marco and Del Bimbo, Alberto},
title = {Restoration of Analog Videos Using Swin-UNet},
year = {2022},
isbn = {9781450392037},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3503161.3547730},
doi = {10.1145/3503161.3547730},
abstract = {In this paper we present a system to restore analog videos of historical archives. These videos often contain severe visual degradation due to the deterioration of their tape supports that require costly and slow manual interventions to recover the original content. The proposed system uses a multi-frame approach and is able to deal also with severe tape mistracking, which results in completely scrambled frames. Tests on real-world videos from a major historical video archive show the effectiveness of our demo system.},
booktitle = {Proceedings of the 30th ACM International Conference on Multimedia},
pages = {6985–6987},
numpages = {3},
keywords = {old videos restoration, analog videos, unet, swin transformer},
location = {Lisboa, Portugal},
series = {MM '22}
}