Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix gemma model conversion #7992

Closed
wants to merge 1 commit into from
Closed

Conversation

Galunid
Copy link
Collaborator

@Galunid Galunid commented Jun 18, 2024

fixes #7964

Untested, since I cannot download gemma without going through license hoops.

@0wwafa Would you mind giving it a try?

@Galunid Galunid added Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix python python script changes labels Jun 18, 2024
@0wwafa
Copy link

0wwafa commented Jun 18, 2024

hmm I solved that by commenting a line in gguf writer...
it seems that in gemma there are multiple chat-templates.
by commenting the line when it raises an exception when there is a duplicate key I cause the last one to overwrite the previous one.

alternatively, I could take the first one.
I don't understand your patch...

@0wwafa
Copy link

0wwafa commented Jun 18, 2024

about the hoops: a single click on huggingface doesn't seem much of a hoop.

@bartowski1182
Copy link
Contributor

@0wwafa there aren't 2 chat templates, it's just being set twice accidentally because 2 places do it, and Gemma happens to be the only one that's calling both of those places

@MoonRide303
Copy link

Gemma 2 models were just released on HF. Trying to convert gemma-2-9b-it results in:

INFO:hf-to-gguf:Loading model: gemma-2-9b-it
ERROR:hf-to-gguf:Model Gemma2ForCausalLM is not supported

@Galunid
Copy link
Collaborator Author

Galunid commented Jun 27, 2024

See #8156

@Galunid
Copy link
Collaborator Author

Galunid commented Jun 29, 2024

This is no longer a problem on master

@Galunid Galunid closed this Jun 29, 2024
@Galunid Galunid deleted the fix-gemma branch June 29, 2024 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
python python script changes Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error converting gemma-1.1-7b-it to gguf.
4 participants