Bug: convert-hf-to-gguf.py fails for Gemma models #7897
Labels
bug
Something isn't working
low severity
Used to report low severity bugs in llama.cpp (e.g. cosmetic issues, non critical UI glitches)
What happened?
When running the convert-hf-to-gguf.py script for the gemma-1.1-2b-it model I get the following error I added to the relevant log output field.
For reproduction of the error, run the script for any Gemma model e.g.:
I already figured out what the problem is: in
set_vocab()
of theGemmaModel
classspecial_vocab.add_to_gguf()
is called twice, once at the beginning of the method inside ofself._set_vocab_sentencepiece()
and then again at the end ofset_vocab()
. Because of this, the chat template is added twice to theGGUFWriter
which raises an exception in theadd_key_value()
method ofGGUFWriter
(in gguf_writer.py) at the second call as'tokenizer.chat_template'
is already present inkv_data
andadd_key_value()
contains the following check:My own quick fix was to remove this check, but I am not sure if this is the proper fix or if
set_vocab()
of theGemmaModel
class should be adjusted, so thatspecial_vocab.add_to_gguf()
is called only once.Name and Version
$ python convert-hf-to-gguf.py
What operating system are you seeing the problem on?
Linux
Relevant log output
The text was updated successfully, but these errors were encountered: