If you use this code for your research, please consider citing:

@article{PADILLA2023116969,
title = {Change detection in moving-camera videos with limited samples using twin-CNN features and learnable morphological operations},
journal = {Signal Processing: Image Communication},
volume = {115},
pages = {116969},
year = {2023},
issn = {0923-5965},
doi = {https://doi.org/10.1016/j.image.2023.116969},
url = {https://www.sciencedirect.com/science/article/pii/S0923596523000516},
author = {Rafael Padilla and Allan F. {da Silva} and Eduardo A.B. {da Silva} and Sergio L. Netto},
}

Download the paper preprint version or published version (available until June, 2023).

---

Lightweight Change Detection in Moving-Camera Videos Using Twin-CNN Features and Learnable Morphological Operations.

• Overview
• Requirements
• Train the model from scratch
• Testing
• Results
  • Metrics
  • Frame-level results
  • Object-level results
• Acknowledgement

Overview

The anomaly detection network consists of five modules as illustrated below:

Where:

(a) aligned reference frame
(b) target frame (where the anomaly is present)
(c) output of the dissimilarity module (DM)
(d) output of the temporal consistency module (TCM)
(e) eroded version of (d) computed by differentiable morphology module (MM)
(f) eroded version of (e) computed by differentiable morphology module (MM)

Requirements

Installing packages and libraries:

If you use conda/anaconda, use the file environment.yml to install the needed packages to run the network: conda env create -f environment.yml

Replicating our results

You can train the model from scratch, or use the pretrained model to evaluate the VDAO testing videos.

Optionally, you could download the full not aligned videos from the VDAO official web site: training set and testing set.Thus, you could apply alternative alignment/registration techniques and train the DN-LMO model.

A. (Optional) Train the model from scratch:

Use the commands below to download the datasets (training + validation) and train the network:

1. Download the aligned dataset for training: sh download_training_dataset.sh
2. Download the aligned dataset for validation: sh download_testing_dataset.sh
3. Train the network: python train.py --fold 1 --net DM_MM_TCM_CM
Notice: Use the argument --fold to select which fold to train (options: --fold 1, --fold 2, --fold 3, --fold 4, --fold 5, --fold 6, --fold 7, --fold 8, --fold 9).

B. Testing

1. Download the aligned dataset for testing: sh download_testing_dataset.sh
2. Download the pretrained model: sh download_pretrained.sh
3. Evaluate the network: sh evaluate.sh

Results

For table of results, videos and frames in the testing set, access the folder /results.

Metrics

Click to expand

Our results are compared against previous works in the same database using the true positive rate (TPR), false positve rate (FPR) and DIS, which is the minimum distance of an operating point to the point of ideal behaviour of a ROC curve, as illustrated below

The best value is obtained when TPR=1 and FPR=0 resulting in a DIS=0, which represents the best possible classification. And it is computed as:

Frame-level results

	average			overall
	TPR	FPR	DIS	TPR	FPR	DIS
DAOMC	0.88	0.39	0.49	0.89	0.42	0.43
ADMULT	0.76	0.36	0.59	0.78	0.39	0.44
MCBS	1.00	0.83	0.83	1.00	0.98	0.98
mcDTSR	0.88	0.25	0.36	0.88	0.28	0.30
CNN+RF	0.74	0.25	0.48	0.75	0.27	0.37
DL-DMO (ours)	0.85	0.18	0.33	0.86	0.21	0.25

Object-level results

	average			overall
	TPR	FPR	DIS	TPR	FPR	DIS
DAOMC	0.81	0.42	0.53	0.82	0.42	0.45
ADMULT	0.70	0.29	0.54	0.72	0.29	0.40
MCBS	0.88	0.83	0.86	0.89	0.83	0.84
mcDTSR	0.86	0.29	0.39	0.86	0.29	0.32
DL-DMO (ours)	0.85	0.22	0.35	0.86	0.22	0.26

Acknowledgement

Most of the experimental results reported in this work were obtained with a Titan X Pascal board gently donated by the NVIDIA Corporation.