You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi!
I have tried Internvideo2-1B-clip in the action recognition task on K400 dataset, I try to use the model without the dataset class you designed.
So what I do in vision is catching 8 frames from video, transform it using test_transform, feed the processed clip into the vision encoder to get the 1x768 feature.
In text I just use the k400_categories.txt and kinetics_prompt you offered, after the text encoder it's 400x16x768 features.
Finally I get these two features in get_sim, and get a rank of the categories, but the result is very bad.
the answer is always not in the top5 choices, the model seems to randomly rank the categories.
I don't know if there is any wrong.
the model I use is chinese_alpaca_lora_7b, InternVideo2-stage2_1b-224p-f4.pt, internvl_c_13b_224px.pth, InternVideo2_CLIP_1B.pth.
The text was updated successfully, but these errors were encountered:
zhengrongz
changed the title
Wrong
Wrong results in Action Recognition task.
Jun 1, 2024
Hi!
I have tried Internvideo2-1B-clip in the action recognition task on K400 dataset, I try to use the model without the dataset class you designed.
So what I do in vision is catching 8 frames from video, transform it using test_transform, feed the processed clip into the vision encoder to get the 1x768 feature.
In text I just use the k400_categories.txt and kinetics_prompt you offered, after the text encoder it's 400x16x768 features.
Finally I get these two features in get_sim, and get a rank of the categories, but the result is very bad.
the answer is always not in the top5 choices, the model seems to randomly rank the categories.
I don't know if there is any wrong.
the model I use is chinese_alpaca_lora_7b, InternVideo2-stage2_1b-224p-f4.pt, internvl_c_13b_224px.pth, InternVideo2_CLIP_1B.pth.
The text was updated successfully, but these errors were encountered: