This repository is for the paper Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models
In this paper, we introduce Self-Checker, a plug-and-play framework that harnesses LLMs for efficient and rapid fact-checking in a few-shot manner. We also present the BingCheck dataset, specifically designed for fact-checking texts generated by LLMs. Empirical results demonstrate the potential of Self-Checker in the use of LLMs for fact-checking. Compared to state-of-the-art fine-tuned models, there is still significant room for improvement, indicating that adopting LLMs could be a promising direction for future fact-checking research.
bash scripts/get_data.sh
cd scripts
# generate knowledge database for Fever
python write_fever_db.py
# generate knowledge database for WiCE
python write_wice_db.py
cd scripts
# Fever
python experiment_fever.py --similar --consistency
# WiCE
python experiment_wice.py --split
# BingCheck
python experiment_bingcheck.py
cd scripts
# Eval Fever Results
python evaluate_fever.py --file_path {fever_result_file}
# Eval WiCE Results
python evaluate_fever.py --file_path {wice_result_file}
# Eval BingCheck Results
python evaluate_fever.py --file_path {bingcheck_result_file}