Confused about valid supervised

[leijue222]

For a image, we do not know how many humans or objects it have. Therefore,

1. The detector may have detected 5 objects (H1, H2, O1, O2, O3), forming 6 valid pairs. But there are only 3 groups of effective pairs （H1-O1, H1-O2,H1-O3） with supervision.
2. Assuming the same number of estimated pairs and supervised pairs, but the order is unknown, how to ensure that the supervision of H1O1 is H1O1, not H1O2 or H2O3.

I would like to know how you do effective supervised learning for these situations.

[fredzzhang]

Hi @leijue222,

There could be more than one way to associate the proposals with ground truth. In this work, we used the same strategy as SCG, which is inherited from object detection.

Specifically for HOIs, you would compute the intersection over union (IoU) between the human boxes of the detected pairs and ground truth pairs, as well as the object boxes. The minimum of these two values will be used as the IoU between the detected pair and ground truth pair. If a detected pair has an IoU of over 0.5 with any ground truth, it is labelled as positive, otherwise negative.

Fred.