NLP - Finding the most wrong false positives / negatives

[ValentinLibouton]

Hello everyone!

I have a question about how to visualize the worst false positives and false negatives.
We have:

# Create DataFrame with validation sentences, validation labels and best performing model prediction labels + probabilities
val_df = pd.DataFrame({"text": val_sentences,
                       "target": val_labels,
                       "pred": model_6_pretrained_preds,
                       "pred_prob": tf.squeeze(model_6_pretrained_pred_probs)})
val_df

# Find the wrong predictions and sort by prediction probabilities
most_wrong = val_df[val_df["target"] != val_df["pred"]].sort_values(by="pred_prob", ascending=False)
most_wrong

most_wrong gives the false positives at the top of the list, but at the bottom we don't have the high probability false negatives but the lowest probability.

most_wrong.tail()

Isn't it better to do it afterwards?:

# split false neg / pos
most_wrong_false_negatives = most_wrong[most_wrong["pred"] == 0]
most_wrong_false_positives = most_wrong[most_wrong["pred"] == 1]

# Check the false positives (model predicted 1 when should've been 0)
for row in most_wrong_false_positives[:10].itertuples():
    _, text, target, pred, pred_prob = row
    print(f"Target: {target}, Pred: {pred}, Prob: {pred_prob}")
    print(f"Text:\n{text}\n")
    print("----\n")
    ```
```python
# Check the false negatives (model predicted 0 when should've been 1)
for row in most_wrong_false_negatives[:10].itertuples():
    _, text, target, pred, pred_prob = row
    print(f"Target: {target}, Pred: {pred}, Prob: {pred_prob}")
    print(f"Text:\n{text}\n")
    print("----\n")

The order of results is not at all the same for false negatives and seems more coherent to me, doesn't it?