mt5-base model (unicamp-dl/mt5-base-en-msmarco) throws 'nan' outputs

[cramraj8]

Hi,

I was running unicamp-dl/mt5-base-en-msmarco: ['▁no' , '▁yes'] model for both English and other My.TyDi languages, but the output scores are nan. When I switched to unicamp-dl/mt5-13b-mmarco-100k: ['▁', '▁true'] model, I get actual logits. I wonder if there's any issues with underlying unicamp-dl/mt5-base-en-msmarco model.

Thanks.

[rodrigonogueira4]

Hi @vjeronymo2 @lhbonifacio any ideas of what could be the problem here?

[lhbonifacio]

Hey @cramraj8,
Thank you for your interest in our work.
Regarding unicamp-dl/mt5-13b-mmarco-100k, the prediction tokens are indeed ['▁', '▁true'] (as reported here).
Maybe @vjeronymo2 can give us more details about that, but as you already mentioned, it is working fine.

For the unicamp-dl/mt5-base-en-msmarco model, I just tested it using the reranking implementation from InPars and it seems to be working fine with the prediction tokens ['▁no' , '▁yes'].
Maybe you could try to use your reranking code that is available here: https://github.com/zetaalphavector/InPars/blob/master/inpars/rerank.py

Thanks.