DAIN-APP Application

This is the source code for the video interpolation application Dain-App, developed on top of the source code of DAIN Dain GIT Project

Table of Contents

1. Introduction
2. Citation
3. Requirements and Dependencies
4. Installation
5. Running application with interface
6. Running application with command line
7. Slow-motion Generation
8. Training New Models
9. Google Colab Demo

Introduction

Dain-App comes with a user interface and a command line script to help new users to start using it with little change to the code. You can also get the Windows binary from the build section.

You can see a few results from those Youtube videos. Turning animations to 60FPS. Turning Sprite Art to 60FPS. Turning Stop Motion to 60FPS. Turning ANIME P1 to 60FPS. Turning ANIME P2 to 60FPS.

Citation

If you find the code and datasets useful in your research, please cite:

@article{Dain-App,
	title={Dain-App: Application for Video Interpolations},
	author={Gabriel Poetsch},
	year={2020}
}
@inproceedings{DAIN,
    author    = {Bao, Wenbo and Lai, Wei-Sheng and Ma, Chao and Zhang, Xiaoyun and Gao, Zhiyong and Yang, Ming-Hsuan}, 
    title     = {Depth-Aware Video Frame Interpolation}, 
    booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
    year      = {2019}
}
@article{MEMC-Net,
     title={MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement},
     author={Bao, Wenbo and Lai, Wei-Sheng, and Zhang, Xiaoyun and Gao, Zhiyong and Yang, Ming-Hsuan},
     journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
     doi={10.1109/TPAMI.2019.2941941},
     year={2018}
}	

Requirements and Dependencies

• numba=0.51.2
• numpy=1.19.2
• opencv-python=4.4.0.46
• pillow=8.0.1
• pyqt5=5.15.1
• python=3.8.5
• scikit-learn=0.23.2
• scipy=1.5.4
• torch=1.7.0+cu110
• torchvision=0.8.1+cu110
• tqdm=4.51.0
• ffmpeg

Installation

If you use Linux. You'd better use conda env. The environment by requirements.txt has been tested on Ubuntu 18.04 and Arch Linux.

conda create --name DAIN-APP --file requirements.txt
conda activate DAIN-APP
pip install 'opencv-contrib-python==4.4.0.46'

Some tips:

• Install pyqt5 by pip may cause crash issue. If you meet this error:

  QObject::moveToThread: Current thread (0x55c1b72bc630) is not the object's thread (0x55c1b72f1eb0). Cannot move to target thread (0x55c1b72bc630)

try to install pyqt by conda instead.

pip uninstall pyqt5
pip uninstall pyqt5-sip
conda install pyqt

Check out the Colab code:

Then, run the script below while inside Dain-App dir to compile the remaining modules. For more details, please refer Installation of DAIN README Doc.

$ ./build-app.sh

Running application with interface

$ python my_design.py

Or run the script to automatically initialize conda and start the GUI

$ ./GUI.sh

Running application with command line

You can see all commands for CLI using this code:

python my_design.py -cli -h

A example of a working code:

python  my_design.py -cli --input "gif/example.gif" -o "example_folder/" -on "interpolated.gif" -m "model_weights/best.pth" -fh 3 --interpolations 2 --depth_awarenes 0 --loop 0 -p 0 --alpha 0 --check_scene_change 10 --png_compress 0 --crf 1 --pixel_upscale_downscale_before 1 --pixel_downscale_upscale_after 1 --pixel_upscale_after 1 --mute_ffmpeg 1 --split_size_x -1 --split_size_y -1 --split_pad 150 --half 0 --step_extract 1 --step_interpolate 1 --batch_size 1 --use_benchmark 0 --force_flow 1 --smooth_flow 0 --downscale -1 --fast_mode 0

Training New Models

Currently Dain-App training code is broken, to train new models, use the DAIN github and import the models to Dain-App

Google Colab Demo

Contact

Gabriel Poetsch

License

See MIT License