Hate Speech Detection with Transformers (BERT & XLNet)

Objective

With accordance to general rules of social media platforms, some posts which include hate-speech are prohibited. This phenomenon is more important in twitter as users can stay ananymose only sharing a line of words. In this project two of the most common Transformer models Bidirectional Encoder Representations from Transformers (BERT) and XLNet.

Results

Results are grouped under the appropriate model.

Model Name	Cost Curve	Performance Curve
BERT
XLNet

Model Name	Data Partition	Accuracy	Precision	Recall	F1-Score
BERT	Train	0.70	0.65	0.95	0.65
BERT	Test	0.48	0.44	0.97	0.61
XLNet	Train	0.70	0.60	0.88	0.70
XLNet	Test	0.56	0.49	0.84	0.62

Resources

• BERT in Pytorch
• XLNet in Pytorch
• Hateful Conduct Policy of Twitter