README.md

🦜 Ask-Anything

| | | |
End2end ChatBOT for video and image.
Explicit communication with ChatGPT.

中文 README 及 中文交流群 | Paper

🚀: We update video_chat by instruction tuning for image & video chatting now! Old version of video_chat moved to video_chat_with_chatGPT. We release instruction data at InternVideo

⭐️: We are also working on a updated version, stay tuned!

🎥 Online Demo [click here]

intro.mp4

🔥 Updates

• 2023/05/11 End-to-end VideoChat

  • VideoChat: Instruction tuning for image & video chatting.

• 2023/04/25 Watch videos longer than one minute with chatGPT

  • VideoChat LongVideo: Incorporating langchain and whisper into VideoChat.

• 2023/04/21 Chat with MOSS

  • VideoChat with MOSS: Explicit communication with MOSS.

• 2023/04/20: Chat with StableLM

  • VideoChat with StableLM: Explicit communication with StableLM.

• 2023/04/19: Code release & Online Demo

  • VideoChat with ChatGPT: Explicit communication with ChatGPT. Sensitive with time. demo is avaliable!
  • MiniGPT-4 for video: Implicit communication with Vicuna. Not sensitive with time. (Simple extension of MiniGPT-4, which will be improved in the future.)

💬 Example

cxk_online.mp4

🔨 Getting Started

Build video chat with:

• ChatGPT
• StableLM
• MiniGPT-4
• MOSS

📄 Citation

If you find this project useful in your research, please consider cite:

@article{2023videochat,
  title={VideoChat: Chat-Centric Video Understanding},
  author={KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao},
  journal={arXiv preprint arXiv:2305.06355},
  year={2023}
}

⏳ Ongoing

Our team constantly studies general video understanding and long-term video reasoning:

• Strong video foundation model.
• Video-text dataset and video reasoning benchmark.
• Video-language system with LLMs.
• Artificial Intelligence Generated Content (AIGC) for Video.
• ...

We are hiring researchers, engineers and interns in General Vision Group, Shanghai AI Lab. If you are interested in working with us, please contact Yi Wang (wangyi@pjlab.org.cn).