Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VLM Support for vLLM Deployment #179

Merged
merged 5 commits into from
Sep 25, 2024
Merged

Conversation

movchan74
Copy link
Contributor

Summary:
This update adds support for Vision-Language Models (VLM) to vLLM deployment, along with necessary tests and documentation updates. Only images are supported at the moment. A bug with insufficient resources error has also been addressed.

Key Changes:

  1. VLM Support for Images: Added Vision-Language Model (VLM) support for vLLM deployment, currently limited to image processing.
  2. Tests for VLM: Introduced tests for VLM on images to ensure proper functionality in vLLM deployments.
  3. Documentation Update: Updated the vLLM deployment documentation to reflect new features and usage details.
  4. Bug Fix: Resolved an issue where deployments could fail due to insufficient resource error because it was restarted.

Note:
Currently, there is no resizing for images in the deployment since vLLM doesn't allow passing arguments into the image processor (see vLLM Issue #7861). It seems the vLLM team is working on this. If needed, we can implement our own preprocessor for image resizing or resize the images before feeding them into the vLLM deployment.

Closes: #170

@movchan74 movchan74 changed the title VLM Support and Bug Fix for vLLM Deployment VLM Support for vLLM Deployment Sep 18, 2024
@movchan74 movchan74 self-assigned this Sep 18, 2024
@movchan74 movchan74 marked this pull request as ready for review September 18, 2024 13:57
Copy link
Contributor

@appoose appoose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine, could reproduce results ( have not checked complex videos though ). But let us merge it.

@movchan74 movchan74 merged commit 1ca768e into main Sep 25, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Vllm multi modal deployment support.
2 participants