Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to read from model.weights.h5 - might be a directory, or too small? #46

Closed
charbull opened this issue Feb 25, 2024 · 3 comments
Closed
Labels
type:support Support issues

Comments

@charbull
Copy link

Hi,

I am experiencing the follow issue, I tried the following versions:
https://www.kaggle.com/models/keras/gemma/frameworks/Keras/variations/gemma_2b_en/versions/1
https://www.kaggle.com/models/keras/gemma/frameworks/Keras/variations/gemma_2b_en/versions/2

/gemma \                                         
--tokenizer vocabulary.spm \
--weights model.weights.h5 \
--compressed_weights 2b-pt-sfp.sbs  --model 2b-pt --verbosity 2
Cached compressed weights does not exist yet (code 256), compressing weights and creating file: 2b-pt-sfp.sbs.
Abort at /Users/charbel/Downloads/gemma/gemma.cpp/./gemma.cc:138: Failed to read from model.weights.h5 - might be a directory, or too small?
zsh: abort      ./gemma --tokenizer vocabulary.spm --weights model.weights.h5  2b-pt-sfp.sbs 

Any ideas how to resolve?

Cheers,
Charbel

@austinvhuang
Copy link
Collaborator

austinvhuang commented Feb 25, 2024

Hi, in general you shouldn't use the --weights parameter at this time.

In the future you can use it to load fine tuned weights and make compressed versions, but that requires a python script to convert weights #11

Instead, only use the compressed weights (you don't need the keras weights, just the sfp files from the GemmaCpp download page).

Second, you probably want to start with the -it "instruction tuned" models which are more appropriate for interactive use. the -pt "pretrained models" are more of a starting point for fine tuning. So in summary:

  • Download 2b-it-sfp.sbs
  • Run with: this tweak to your command (remove --weights, pt->it):
./gemma \                                         
--tokenizer vocabulary.spm \
--compressed_weights 2b-it-sfp.sbs  --model 2b-it

and see if that works.

@austinvhuang
Copy link
Collaborator

Closing for now but if you still run into an issue we'll reopen and help.

@charbull
Copy link
Author

Thanks ! that worked, I was using the keras weights.

@tilakrayal tilakrayal added the type:support Support issues label Apr 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:support Support issues
Projects
None yet
Development

No branches or pull requests

3 participants