Skip to content

Repo for our CVPR Paper: Watch your Up-Convolution: CNN Based Generative Deep Neural Networks areFailing to Reproduce Spectral Distributions

License

Notifications You must be signed in to change notification settings

cc-hpc-itwm/UpConv

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

60 Commits
 
 
 
 
 
 
 
 

Repository files navigation

This repository provides the official Python implementation of Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions (Paper: https://arxiv.org/abs/2003.01826).

Common up-convolution methods are inducing heavy spectral distortions into generated images. (Left) Statistics (mean and variance) after azimuthal integration over the power-spectrum of real and GAN generated images. (Right) Results of the same experiments as above, adding our proposed spectral loss during GAN training.

Spectral Regularization

Since common generative network architectures are mostly exclusively using image-space based loss functions, it is not possible to capture and correct spectral distortions directly. Hence, we extend existing GAN architectures in two ways:

  • Spectral loss term
  • Replace last last up-convolution unit.

Dependencies

Tested on Python 3.6.x.

Downloading Dataset

Link to download CelebA dataset.

Training Netwroks

Training vanilla models

We train different GAN models using this repo. Then, we employ our Visualization script to analyse the frequency behaviour.

Training Spectral Regularization models

Make the following changes to incorporate the regularizer. Then, train the model.

  • Substitue train.py for train_spectrum.py
  • Substitue module.py for module_spectrum.py

Click here to go the Regularization implementation.

Deepfake detection

Experiments and code for the deepfake detection parts of the paper can be found in the repository of our prior Arxiv pre-print.

Citation

If this work is useful for your research, please cite our paper:

@misc{durall2020upconv,
    title={Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions},
    author={Ricard Durall and Margret Keuper and Janis Keuper},
    year={2020},
    eprint={2003.01826},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

About

Repo for our CVPR Paper: Watch your Up-Convolution: CNN Based Generative Deep Neural Networks areFailing to Reproduce Spectral Distributions

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published