-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Request for finetuned InternVideo2-1B results on video retrieval benchmarks #136
Comments
Hi! Considering the cost for diverse downstream datasets, we only provide the zero-shot results~ |
@roberto-amoroso where you able to obtain the authors results for MSRVTT (zero shot) ? |
@Andy1621 Thanks for your reply. Yes, I am aware that finetuning the model could be expensive, so I was hoping you have some internal results of your 1B model finetuned on MSRVTT that you could share... thanks anyway |
@nsreeprem do you mean if I was able to reproduce the 0-shot performance presented in Table 9 of the paper? |
@roberto-amoroso yes, I meant to ask if you were able to reproduce the results for zero-shot R@1. I am finding close to 47% (~5% lower) performance (R@1) than what is mentioned in the Table 9. |
@nsreeprem The 0-shot performance I measured on MSRVTT by using the s2-1B model is 51.8 (0.1% lower) for T2V R@1 and 49.3 (1.6% lower) for V2T R@1. These results are obtained by considering the ITM re-ranking stage (i.e., what is called |
@Andy1621 did you use the 9k or 7k MSR-VTT train split for finetuning the 6B model (Table 10)? |
We follow Unmaked Teacher to finetune it the downstream tasks. |
where to get this model? thanks |
The ft performance gap of 1B compared to 6B is close to the zs performance gap, you can estimate it. |
@Andy1621 @leexinhao would you be releasing the hyperparameters for finetuning the 1B or 6B model? |
You could refer to https://github.com/OpenGVLab/unmasked_teacher, we use similar hyperparameters except using deepspeed. |
Are you used the hyperparamers as reported in the config.py? I get a big performance gap between my reproduce version and the InternVideo2-1B-stage2-f4 |
Hi, great work and thanks for releasing the code. In Table 10 of your InternVideo2 paper, you reported the results of finetuning video retrieval in both T2V and V2T on MSR-VTT, LSMDC, DiDeMo, MSVD, ActivityNet, and VATEX for the 6B model.
Could you please provide the results for the finetuned InternVideo2-1B model as well?
This would be very helpful for literature comparisons with models of similar size.
Thanks a lot
The text was updated successfully, but these errors were encountered: