EvaluateQA

Package for evaluate QA datasets and Leaderboard with SOTA approaches

Install

pip install evaluateqa

Supported datasets

Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answering

from evaluateqa.mintaka import evaluate

predictions = {
    '9ace9041': 'Q90',
    '9ace9042': 3,
    ...
}

results = evaluate(
    predictions,
    split='test',
    mode='kg',
    lang='en',
)