MIRACLE

EMNLP2023-findings: MIRACLE: Towards Personalized Dialogue Generation with Latent-Space Multiple Personal Attribute Control

arxiv link

Dataset is uploaded in here.

The Concept

We modeling the multi-facted personality as the fusion of multiple personal attribute ($P_1, P_2, \cdots, P_N$), where each attribute may have many aspects ($p_1, p_2, \cdots, p_m$).

Setup

This repo is build based on 🤗 huggingface transformers and torchdiffeq.

The training and evaluating can be done with a single RTX3090/4090.

The recommendate environment :

conda create -n miracle python=3.9
pip install -r requirements.txt

Training and evaluating

In the following part, we assume you have downloaded the dataset we used

Firstly, you should train two model for evaluation:

• single-personal-attribute text classifier for calculating personalization score.

ATTR=languagestyle bash train_classifier.sh

• NLI model for calculating dialogue resposne semantic coherence.

bash train_nli.sh

Then, you can train our Miracle and generate personalized responses by:

bash pipeline.sh

which call train_cvae.sh for training and gen_cave.sh for generation. We select the 11th epoch model as our final model. Notice that for different dataset or different senarioes, you may need to adjust hyper-parameters to gain better results.

Customize you data

To train our model, you can use any personl-attribute-dense dataset in the format like our released data.

We also upload our ChatGPT-API script as reference. It generate personalized response in aspect level. Notice that you need to prepare you own topics in dataset/sample_topics:

python chatgpt_data.py --key 'Your OpenAI API key' --aspect 'xx1'

With the collected data, you need to format them as follows in dataset/dialogue_yy.jsonl for each attribute:

{"input": ["user post1", "user post2"], "output": ["model resp1", "model resp2"], "tag": 'xx1'}
{"input": ["user post1", "user post2"], "output": ["model resp1", "model resp2"], "tag": 'xx2'}
...

We recommendate you to clean them before use to make sure it's personality richness.

Before training, change the schema.py to set you dataset file paths.

Similar Insights

• Controllable and Compositional Generation with Latent-Space Energy-Based Models. NIPS 2021 code
• Composable Text Controls in Latent Space with ODEs. EMNLP 2022 code
• A Distributional Lens for Multi-Aspect Controllable Text Generation. EMNLP 2022 code
• Controllable Text Generation via Probability Density Estimation in the Latent Space. ACL 2023 code

Citation

@misc{lu2023miracle,
    title={MIRACLE: Towards Personalized Dialogue Generation with Latent-Space Multiple Personal Attribute Control},
    author={Zhenyi Lu and Wei Wei and Xiaoye Qu and XianLing Mao and Dangyang Chen and Jixiong Chen},
    year={2023},
    eprint={2310.18342},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}