This repository contains code for DYPLOC ver-0.1, which is the original code we used for experiments in our ACL 2021 paper submission. If you find our work useful, please cite:
@inproceedings{hua-etal-2021-dyploc,
title = "DYPLOC: Dynamic Planning of Content Using Mixed Language Modelsfor Text Generation",
author = "Hua, Xinyu and
Sreevatsa, Ashwin and
Wang, Lu",
booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP)",
month = aug,
year = "2021",
publisher = "Association for Computational Linguistics",
}
The original project is tested under the following environment:
pytorch==1.5.1
pytorch-lightning==1.0.0
transformers==3.2.0
Please first download data from this link.
trainable/
: datasets ready to be loaded bydataset.py
checkpoints/
: folders to save checkpointstrain.py
: code for training, will save checkpoints tocheckpoints/
and statistics totb_logs/
.generate.py
: code for inference, will save output tooutputs/
.system.py
: code for implementation of the training, validation, data loading, optimization routines.dataset.py
: code for datasetsdyploc.py
: code for model implementation, including decoding specific methods such as nucleus sampling
Our model is trained using the following command.
EXP_NAME="demo_cmv"
CUDA_VISIBLE_DEVICES=0 python train.py \
--exp-name=${EXP_NAME} \
--dataset-name=cmv \
--warmup-steps=1000 \
--max-epochs=5 \
--batch-size=4 \
--fp16 \
--marginalization=tok
For decoding:
EXP_NAME="demo_cmv"
CUDA_VISIBLE_DEVICES=0 python generate.py \
--exp-name=${EXP_NAME} \
--dataset-name=cmv \
--set-type=test \
--system-setup=oracle \
[--quiet \]
--epoch=5 \
--batch-size=64 \
--fp16