We release the roberta-large model that was finetuned on the Is Shuffle task and achieves a >97% accuracy at the task on the standard Wall Street Journal-based test set.
The file name is: cls_roberta-large_supervised_shuffle.bin and an example of usage is provided in run_supervised.py.
The model can be loaded through the HuggingFace library:
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tokenizer = AutoTokenizer.from_pretrained("roberta-large")
model = AutoModelForSequenceClassification.from_pretrained("roberta-large")
coherence_model.load_state_dict(torch.load("cls_roberta-large_supervised_shuffle.bin"))