-
-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Model][LoRA]LoRA support added for glm-4v #10418
Conversation
Signed-off-by: B-201 <Joy25810@foxmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can do one of these:
🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your contribution, LGTM, please add related docs!
ping @DarkLight1337 Could you plz look at this PR,thanks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding support!
Oh, I forgot again that we need to update the Supported Models page. Can you open another PR for that? Thanks. |
Sure, I will open another PR to update the docs. |
Signed-off-by: B-201 <Joy25810@foxmail.com> Signed-off-by: Manjul Mohan <manjul.mohan@ibm.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com> Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Signed-off-by: B-201 <Joy25810@foxmail.com> Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: B-201 <Joy25810@foxmail.com> Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
To facilitate Lora support, the language model and VL model have been separated. I tested it locally, but it seems I encountered the same issue as this Comment, where the results from vllm differ slightly from those of transformers. However, there is no issue with loading the Lora model, and the results appear to be reasonable.