- Precision = positive predictive value = TP / (TP+FP)
Fraction of relevant instances among retrieved instances
- Recall = sensitivity = TP/ (TP + FN)
Fraction of the total amount of relevant instances that were actually retrieved
- Example:
A ML algorithm identifies 8 dogs in a picture where there are 12 dogs and some cats. Of the identified dogs, 5 are really dogs (TP), and there are 3 cats identified as dogs (FP)
Precision = 5/8 From all dogs identified, just 5 were actually dogs
Recall= 5/12 From all the dogs, just 5 were identified
Internet picture.
In the picture we see:
- 1 TP
- 2 FP
- 7 TN
- 0 FN
- Classification accuracy.
Accuracy = nº of correct predictions / total of predictions = (TP+FN)/ total of samples
If 98% of samples are type A, our model can easily get 98% of training accuracy, so it's important to have a well balanced system if we'll trust this metric
- Confusion matrix.
n=165 Predicted NO Predicted YES
Actual NO 50 10
Actual YES 5 100
- TP : predicted YES and actual output was NO
- TN : " " NO " " NO
- FP : " " YES " " NO
- FN : " " NO " " YES
- AUC:
Axis Y: TP rate Axis X: FP rate
TP rate = sensitivity = TP / (FN+TP) FP rate = specificity = FP / (FP+TN)
- F1 score:
F1 = 2 / (1/precision + 1/recall)
It tries to find a balance between both metrics
- MSE:
Is the average of the square difference between the Original Values and the Predicted Values. Due to the ², higher errors are highlighted and we can work them out.
- When using each metric: