Demo script for Internvideo2 model for Video Question Answering or Summarization task

[Varun-GP]

Dear Authors,
How can I use the Internvideo2 model for Video Question Answering or Summarization tasks given a video? Please provide a demo script if any for testing on new videos.

Thanks.

[WeiminLee]

I can not successfully run the demo code posted on huggingface. I did not know why.

code: https://huggingface.co/OpenGVLab/InternVideo2_chat_8B_HD/blob/main/demo.py

error:
File "/data/lwm-data/AwesomeCode/InternVideo2-Chat/demo.py", line 19, in <module> model = AutoModel.from_pretrained( File "/data/lwm-data/lwm-env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 559, in from_pretrained return model_class.from_pretrained( File "/data/lwm-data/lwm-env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3482, in from_pretrained raise EnvironmentError( OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /data/model_hub/InterVideo2_chat.

[yinanhe]

I can not successfully run the demo code posted on huggingface. I did not know why.

code: https://huggingface.co/OpenGVLab/InternVideo2_chat_8B_HD/blob/main/demo.py

error: File "/data/lwm-data/AwesomeCode/InternVideo2-Chat/demo.py", line 19, in <module> model = AutoModel.from_pretrained( File "/data/lwm-data/lwm-env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 559, in from_pretrained return model_class.from_pretrained( File "/data/lwm-data/lwm-env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3482, in from_pretrained raise EnvironmentError( OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /data/model_hub/InterVideo2_chat.

What's your transformers version?

[yinanhe]

You can just follow the readme of InternVideo2-Chat

[WeiminLee]

I can not successfully run the demo code posted on huggingface. I did not know why.
code: https://huggingface.co/OpenGVLab/InternVideo2_chat_8B_HD/blob/main/demo.py
error: File "/data/lwm-data/AwesomeCode/InternVideo2-Chat/demo.py", line 19, in <module> model = AutoModel.from_pretrained( File "/data/lwm-data/lwm-env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 559, in from_pretrained return model_class.from_pretrained( File "/data/lwm-data/lwm-env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3482, in from_pretrained raise EnvironmentError( OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /data/model_hub/InterVideo2_chat.

What's your transformers version?

Thanks, I have successfully run this demo code. Because I can only run it on local linux server. I changed a lot of configurations.

In particular, the model paths for Q-fomer, Bert-base-uncased and Mistral-7B

[yinanhe]

I can not successfully run the demo code posted on huggingface. I did not know why.
code: https://huggingface.co/OpenGVLab/InternVideo2_chat_8B_HD/blob/main/demo.py
error: File "/data/lwm-data/AwesomeCode/InternVideo2-Chat/demo.py", line 19, in <module> model = AutoModel.from_pretrained( File "/data/lwm-data/lwm-env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 559, in from_pretrained return model_class.from_pretrained( File "/data/lwm-data/lwm-env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3482, in from_pretrained raise EnvironmentError( OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /data/model_hub/InterVideo2_chat.

What's your transformers version?

Thanks, I have successfully run this demo code. Because I can only run it on local linux server. I changed a lot of configurations.

In particular, the model paths for Q-fomer, Bert-base-uncased and Mistral-7B

Yes, you need to ensure that your local has the cache of Bert-base-uncased's config and Mistral-7B-v0.3's config. Thank you for your feedback.