Using Simple Transformers for Yes/No QA

[ss2342]

I am trying to train the RoBERTa model on the BOOLQ (boolean questions) dataset which is a Yes/No Question Answering Dataset. It was released by google research test a QA LM's ability to answer naturally occurring yes/no questions. It is currently one of the SuperGLUE tasks as well.

For my purposes, I first fine-tuned a pre-trained RoEBRTa model on the Multi-NLI dataset (as done in the Original BoolQ Paper) using the Simple Transformers library. I then intended to use the fine-tuned model on the BoolQ Dataset using the QA LM capabilities of Simple Transformers. However, I noticed that for QA, the data must be structured to have the answer_start as well which indicates where the answer appears in the context. However, since BoolQ is not an Extractive QA task, it does not have the answer_start index available in the dataset. The BoolQ dataset contains a Passage (Context), a Question, and an Answer.

These are some examples from the BoolQ dataset:

I was wondering if there is any way to still use the Simple Transformers library for Yes/No QA if the answer_start index is not available. Thank you!

I am attaching some links here for more info on the BoolQ dataset:
BoolQ Paper: https://arxiv.org/abs/1905.10044
BoolQ Dataset: https://github.com/google-research-datasets/boolean-questions
BoolQ on SuperGLUE: https://super.gluebenchmark.com/leaderboard