Link to paper:
Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments. arXiv preprint arXiv:2406.11370.
Run source install.sh
to quickly create an virtual environment named ZEPO and install all dependencies.
source install.sh
Setup the OPENAI API for calling GPT-3.5-turbo.
export OPENAI_API_KEY=[YOUR_KEY]
We provide a default script to reproduce our main experiments in optimizing the prompts for COH-SummEval.
sh run_zepo.sh
-
Design your own prompts: We provide default instruction to be optimized at
init_prompts.json
and the prompt template for corresponding tasks atprompts.py
. It is interesting to explore different initialization prompts and inspect the fairness. -
Use your own datasets: Feel free to check the data formats that we have implemented in
\data
, and it is easy to adapt other datasets to the same format and implement your own dataloader for conducting pairwise comparisons. -
Support more advanced LLM prompt optimizers: We have only studied a basic LLM optimizer to paraphrase the prompt and do greedy search in
zepo.py
. Having more advanced LLM optimizers can enable more exploitation-driven prompt search in more diverse instruction formats. -
ZEPO mitgates biases Ideally, given a sufficient number of random sampling in pairwise comparisons, ZEPO can always optimize toward fairness, such that all representative biases can be mitigated in one goal, including position bias, verbosity bias, and self-preference bias. We welcome all extension study with respect to the impact of these biases and the extent that they are mitigated by ZEPO.
We acknowledge that some scripts were modified based on PairS.
If you find our work to be useful, please cite:
@article{zhou2024fairer,
title={Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments},
author={Zhou, Han and Wan, Xingchen and Liu, Yinhong and Collier, Nigel and Vulić, Ivan and Korhonen, Anna},
journal={arXiv preprint arXiv:2406.11370},
year={2024}
}