Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added PaliGemma #2683

Merged
merged 6 commits into from
May 31, 2024
Merged

Added PaliGemma #2683

merged 6 commits into from
May 31, 2024

Conversation

JosselinSomervilleRoberts
Copy link
Contributor

No description provided.

@JosselinSomervilleRoberts
Copy link
Contributor Author

Not sure how to deal with the dependency issue. PaliGemma needs transformers 4.41 so I just installed it even with the conflicts and it ran fine.

teetone
teetone requested changes May 28, 2024
View reviewed changes
src/helm/clients/vision_language/paligemma_client.py Outdated Show resolved Hide resolved
src/helm/config/model_deployments.yaml Show resolved Hide resolved
src/helm/config/model_metadata.yaml Outdated Show resolved Hide resolved
src/helm/clients/vision_language/paligemma_client.py Outdated Show resolved Hide resolved
@teetone teetone self-requested a review May 31, 2024 03:51
@teetone
Copy link
Member

teetone commented May 31, 2024

I think we're good to merge once the checks pass.

@teetone
Copy link
Member

teetone commented May 31, 2024

Not sure how to deal with the dependency issue. PaliGemma needs transformers 4.41 so I just installed it even with the conflicts and it ran fine.

Following up in a separate issue, since the code for the models is self-contained.

@teetone teetone added models VHELM Holistic Evaluation of Vision-Language Models (VLM) labels May 31, 2024
@teetone teetone merged commit 2436fe9 into main May 31, 2024
6 checks passed
@teetone teetone deleted the joss-paligemma branch May 31, 2024 04:33
xuwangyin pushed a commit to xuwangyinx/helm that referenced this pull request Jun 23, 2024
@JosselinSomervilleRoberts @teetone @xuwangyin
Added PaliGemma (stanford-crfm#2683)
8f442b4 
Co-authored-by: Tony Lee <tonylee@meta.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@teetone teetone teetone approved these changes

Assignees
No one assigned
Labels
models VHELM Holistic Evaluation of Vision-Language Models (VLM)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@JosselinSomervilleRoberts @teetone