What exactly is object pointer?

[jeezrick]

Hi, I would appreciate it if someone could shed some light on the object pointer feature introduced in SAM2.

As mentioned in the paper:
In addition to the spatial memory, we store a list of object pointers as lightweight vectors for high-level semantic information of the object to segment, based on mask decoder output tokens of each frame.

However, in the (inference) code, I just don't see the benefit of this particular token. Moreover, the tracking still seems to work if you disable it. I wonder if someone could share their insights on this.

[jeezrick]

and of course I see the ablation experiment.

[13512263278]

the object pointer vector for the output mask, extracted based on the output token from the SAM mask decoder.