Should we use "DetectionIoUEvaluator" to evaluate the detection model performance in a OCR task?

[bdeng3]

Hi Team,

I got a question on whether we should use DetectionIoUEvaluator to evaluate PaddleOCR's DB Detection model.

If I understand correctly, DB algorithm does not have "region proposal" and thus, there is no non-maximum suppression or IoU threshold involved during the inference stage. Instead, if we wanted DB to output more bounding boxes, we need to lower the "det_db_box_thresh" parameter.

So it seems to me if we wanted to draw a PR-curve to evaluate the detection performance, we need to adjust the "det_db_box_thresh" to balance the precision & recall. While DetectionIoUEvaluator offers no such options, it does have a "iou_constraint" parameter, but it seems to me the IoU threshold does not play a role here when evaluating DB detection.

What's your thought on this? Any suggestion is welcome

[LDOUBLEV]

Actually, for object detection, you are absolutely right.

For text detection, even if your iou reaches 0.8, it is still possible to miss a character.

Also, there may be curved text in the image, not all horizontal text.