FactCheck Editor

Translate data

To translate data use translate_data.py.
- Usage:
  - Claim detection test split: python -m scripts.translate_data --task claim_detection --split test
  - Veracity prediction test split: python -m scripts.translate_data --task veracity_prediction --split test

Setup

Make a copy of the .env.example file in `.env``
Contact Factiverse for obtaining the values for SERVER_ENDPOINT, AUTH0_AUDIENCE, AUTH0_TOKEN_URL, AUTH0_CLIENT_ID, AUTH0_SECRET
Setup ollama following guide at ollama.ai and specify the endpoint for OLLAMA_HOST
Specify OpenAI credentials deployed in Azure for AZURE_OPENAI_KEY, AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_MODELID

Evaluate claim detection

First run claim detection predictions using python -m code.claim_detection.claim_detection
Then run the plotting script python -m scripts.claim_detection_plots

Evaluate veracity prediction

First run veracity prediction: python -m code.veracity.veracity_prediction
Then run veracity eval: python -m code.veracity.nli_eval
Then run the plotting script: python -m scripts.veracity_plots