Skip to content

"A Deep Moving-camera Background Model" [Erez, Shapira Weber, and Freifeld, ECCV 2022]

License

Notifications You must be signed in to change notification settings

BGU-CS-VIL/DeepMCBM

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

52 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DeepMCBM

Authors: Guy Erez, Ron Shapira Weber, and Oren Freifeld.

This code repository corresponds to our ECCV '22 paper: DeepMCBM: A Deep Moving-camera Background Model. DeepMCBM is a novel 2D-based method for unsupervised learning of a moving-camera background model, which is highly scalable and allows for relatively-free camera motion.

Table of Contents
  1. Documentation
  2. Results
  3. Visual Comparisons

Documentation

Environment

The repository is equipped with a DeepMCBM_env.yml file.
Run conda env create -f DeepMCBM_env.yml from your terminal to set a conda environment using this file.
To ensure the environment is set properly, activate the new environment and run a "dry run" with few epochs:

conda activate DeepMCBM
python src/DeepMCBM.py --DryRun

Train, Predict and Evaluate

To train, predict and evaluate a deepMCBM module on the default tennis sequence:

python src/DeepMCBM.py 

Input, Output and Checkpoints

The default values for the input, output, and checkpoints paths are set in src/args.py and can be changed to any path you wish. The requirement for the input directory is to have the following subdirectories: "frames" include the sequence frames, and if ground truth labels are available, a "GT" directory containing the ground truth frames. See the input/tennis for an example. The output directories are named by the sequence and the log_name argument: output/sequence_name/log_name in this directory you will find:

  • background_estimation directory containing the background estimation of the model.
  • MSE directory containing the Mean Square Error (MSE) computed using the ground truth labels.
  • panoramic_robust_mean.png image, shows the alignment result of the STN module.

You can change the log_name simply by adding log_name "my_new_name" to your command line.

Predict Using a Pretrained Model

To only predict and evaluate metrics:

python src/DeepMCBM.py --no_train_BMN --no_train_STN 

You can change the loaded checkpoint using a flag:

python src/DeepMCBM.py --no_train_BMN --no_train_STN --BMN_ckpt ckpt_file.ckpt  

Or by editing the MCBM_CKPT argument in src/args.py

Note: when using a pretrained model, the argument --pad, describing the size of the padding, must be the same as in the training phase.

Results

tennis.mp4
dog-gooses.mp4
flamingo.mp4
horsejump-high.mp4
stunt.mp4
zoomInZoomOut.mp4
bmx-trees.mp4

Visual Comparisons

tennis.pdf

flamingo.pdf

dog-gooses.pdf

bmx-trees.pdf

horsejump-high.pdf

About

"A Deep Moving-camera Background Model" [Erez, Shapira Weber, and Freifeld, ECCV 2022]

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages