Skip to content

StanLei52/ViT-Lens-Integration

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 

Repository files navigation

ViT-Lens Integration with Multimodal Foundation Models

What's New: 🎉

  • Release 3D ViT-Lens Integration with InstructBLIP.
  • Release 3D ViT-Lens Integration with SEED-LLaMA.

Usage / Example

ViT-Lens with InstructBLIP

  • Installation:
    1. Install ViT-Lens following the official repo.
    2. Installation for InstructBLIP
      cd vitlens_instructblip/
      pip install -e .
  • Example
    cd vitlens_instructblip/
    python vitlens_instblip_example.py

Paper and Citation

You can find more details in our paper.

If you're using LAVIS in your research or applications, please cite using this BibTeX:

@InProceedings{Lei_2024_CVPR,
    author    = {Lei, Weixian and Ge, Yixiao and Yi, Kun and Zhang, Jianfeng and Gao, Difei and Sun, Dylan and Ge, Yuying and Shan, Ying and Shou, Mike Zheng},
    title     = {ViT-Lens: Towards Omni-modal Representations},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {26647-26657}
}

Contact us

If you have any questions, comments or suggestions, please create an issue or reach us at leiwx52@gmail.com.

About

ViT-Lens Integration to Multimodal Foundation Models

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages