Fix Grounding DINO nan when class tokens exceeds 256

[Divadi]

Motivation

Grounding DINO tokenizes class names using GLIP code, and associates tokens with original class names. However, "max_num_entities" is capped at 256.

mmdetection/mmdet/models/detectors/glip.py

Lines 98 to 100 in fe3f809

	def create_positive_map(tokenized,
	tokens_positive: list,
	max_num_entities: int = 256) -> Tensor:

This causes issues when Grounding DINO is run (fine-tune or zero-shot inference) with a large number of class names whose total number of tokens exceed this amount. In my use case, 100~ classes ended up yielding 280 tokens. This causes nans in the prediction scores, because later class names are not associated with any tokens.

Modification

Simply set the max_num_entities as max_text_len defined for Grounding DINO contrastive objective. This does not break original code, as max_text_len by default is 256.

Note that max_tokens for language_model also needs to be updated, or it can silently not detect any classes past 256 tokens. However, that can just be modified in config, unlike max_num_entities.

[CLAassistant]

All committers have signed the CLA.

[hhaAndroid]

@Divadi Thank you