C-Net

C-Net: Contextual Network for Sarcasm Detection
Authors: Amit Kuamr Jena, Aman Sinha, Rohit Agarwal
(Accepted at FigLang 2020 Workshop, co-located with ACL 2020)

Why it is Important ?

1. Classifying an utterance as sarcastic or not in isolation can be futile since most of the time the sarcastic nature of a sentence heavily relies on its context.
2. It plays a crucial role in improving the effectiveness of chatbot systems. Sentiment classification systems can fail in the absence of a robust sarcasm detection system. A sarcastic sentence can express a negative sentiment even with the presence of positive or neutral sentiment words in that sentence.
3. It can make an artificially intelligent agent closer to imitate human behaviour and enable it to better understand true intentions and emotions of humans.

Main Challenge

To use the dialogue context information of an utterance effectively to decide its sarcastic nature.

Methods Used:

1. Baseline methods: Logistic Regression, Naive Bayes, SGD Classifier, XGBoost, SVM, Vanilla RNN, Bi-LSTM.
2. Pre-trained Networks: DeepMoji, ELMo, XLNet, BERT, RoBERTa.
3. C-Net (Our approach).
4. Timestamping.

Baseline methods and Pre-trained networks classified responses in isolation, without any context information. C-Net effectively utilises partial context information. Timestamping utilises complete context information.

C-Net Architecture

Here, ‘n’ is the maximum size of the context set. Model-1, 2, 3... n+1 are BERT (base-uncased) models which are trained separately on the response sentences, last sentence of context sets, second last sentence of context sets and so on till the first sentence of context sets respectively. Probability values generated by these n+1 models are used by the Fusion Layer to generate another probability value as output, which tells about the possibility of sarcasm presence in the response.

1. It uses pseudo-labeling to provide labels for the context sentences by giving them the same label as the response sentence. This is followed by a fusion layer.
2. The fusion layer can be either a Logistic Regression or a Simple Exponential Smoothing model. Since the sequence of dialogues in a conversation matters in deciding the polarity or emotion of future dialogues, simple exponential smoothing was used to take advantage of the sequential nature of the dataset. Simple Exponential Smoothing (SES) is a time series forecasting method for univariate data without a trend or seasonality.

Results

Overall, the C-Net model gave the best results as compared to all the approaches on the twitter test dataset (0.750 F1-score).

Citation

The paper in the ACL anthology can be found here and do cite our work if it proved to be useful in anyway!

@inproceedings{kumar-jena-etal-2020-c,
    title = "{C}-Net: Contextual Network for Sarcasm Detection",
    author = "Kumar Jena, Amit  and
      Sinha, Aman  and
      Agarwal, Rohit",
    booktitle = "Proceedings of the Second Workshop on Figurative Language Processing",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.figlang-1.8",
    pages = "61--66",
    abstract = "Automatic Sarcasm Detection in conversations is a difficult and tricky task. Classifying an utterance as sarcastic or not in isolation can be futile since most of the time the sarcastic nature of a sentence heavily relies on its context. This paper presents our proposed model, C-Net, which takes contextual information of a sentence in a sequential manner to classify it as sarcastic or non-sarcastic. Our model showcases competitive performance in the Sarcasm Detection shared task organised on CodaLab and achieved 75.0{\%} F1-score on the Twitter dataset and 66.3{\%} F1-score on Reddit dataset.",
}