Text-Adaptive Generative Adversarial Network (TAGAN)

A PyTorch implementation of the paper "Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language". This code implements a Text-Adaptive Generative Adversarial Network (TAGAN) for manipulating images with natural language.

Requirements

• PyTorch 0.4
• Visdom
• Pillow
• fastText.py (Note: if you have a problem when loading a pretrained model, try my fixed code)
• NLTK

Pretrained word vectors for fastText

Download a pretrained English word vectors. You can see the list of pretrained vectors on this page.

Datasets

• Oxford-102 flowers: images and captions
• Caltech-200 birds: images and captions

The caption data is from this repository. After downloading, modify CONFIG file so that all paths of the datasets point to the data you downloaded.

Pretrained models

• Oxford-102 flowers: flowers_G.pth
• Caltech-200 birds: birds_G.pth

Please put these files in ./models/ folder.

Run

• scripts/preprocess_caption.sh
  Preprocess caption data using fastText embedding. You only need to run it once before training.
• scripts/train_[flowers/birds].sh
  Train a network. If you want to change arguments, please refer to train.py.
• scripts/test_[flowers/birds].sh
  Test a trained network. After running it, please see ./test/result_[flowers/birds]/index.html.

Results

Citation

Please cite our paper when you use this code.

@inproceedings{nam2018tagan,
  title={Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language},
  author={Nam, Seonghyeon and Kim, Yunji and Kim, Seon Joo},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
  year={2018}
}

Contact

Please contact shnnam@yonsei.ac.kr if you have any question about this work.