-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Training and Evaluation Code for ViClip #131
Comments
Hi! For the zero-shot evaluation, you can refer to the VideoCLIP in InternVideo2. |
@Andy1621 Thanks for the quick response, are you referring to the scripts in InternVideo/InternVideo2/multi_modality/scripts/evaluation/clip/zero_shot, if so, it seems they are for evaluating InternVideo2 clip. Would the scripts and code work off-the-shelf for not ViClip models that you have shared? Do we need to make any changes? It would also be great if you can share the eval code for ViClip directly. |
Hi~ You can find the evaluation sctipets here |
@Andy1621 Thanks for you quick response, will try that to reproduce the results. |
@Andy1621 I tried to do zero-shot eval on msrvtt-1k with scrpts from here |
I think it's a bug when cleaning the code, you can fix it in # from models.vindlu import VindLU
# from models.vindlu_vit import VindLU_VIT
# from models.vindlu_videoclip import VindLU_VideoCLIP
# from models.vindlu_blip_qformer import VindLU_BLIP_QFormer
from models.viclip import ViCLIP And also change the model in |
@Andy1621 Thanks, it solves the problem, however i think the code is still not complete as i get following error: Traceback (most recent call last): |
Did you solve this problem? I got the same error. |
@Code-kunkun Yes, you need to change line 79 in tasks/retrieval_utils.py
Let me know if that works |
Thanks for your quick reply! It works🥳. |
@Andy1621 Thanks for your help so far with the zero-shot evaluation, can you please refer to me which scripts/code to use for full fine-tuning of the ViCLIP models? |
Dear authors,
Great work and thanks for releasing the code for ViClip pretraining on InternVid-10M-FLT. Firstly, It would be really great if the pre-trainning instructions are more detailed, like which clip models to start from, paths for config etc.
Secondlly, can you please also release the evaluation code and scripts for evaluating pretrained ViCLIP models for zero shot kinetics-400, ssv2, ucf etc. I want to reproduce the number for zero-shot evaluation in my local setup.
Thanks and Regards
The text was updated successfully, but these errors were encountered: