-
Notifications
You must be signed in to change notification settings - Fork 225
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Vicuna Models checkpoints transfer script #1657
Conversation
cc: @mattdangerw |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! A couple comments...
Next up, you could try uploading these to your individual user on Kaggle, and making a PR that updates our presets file here -> https://github.com/keras-team/keras-nlp/blob/master/keras_nlp/src/models/llama/llama_presets.py
That would give us all a way to test the vicuna models end to end, then we can copy them to the Keras org on Kaggle when they look good.
Thanks!
print("\n-> Saved the tokenizer") | ||
|
||
# === Upload the preset === | ||
uri = f"kaggle://keras/vicuna/keras/{preset}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's do this like the phi3 script
from keras_nlp import upload_preset # noqa: E402 |
That will still allow people to run this who do not have access to the keras kaggle org.
from keras_nlp.models import LlamaCausalLMPreprocessor | ||
from keras_nlp.models import LlamaTokenizer | ||
|
||
PRESET_MAP = {"vicuna_1.5_7b_en": "lmsys/vicuna-7b-v1.5"} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the weight conversion all the same as llama 2? If so could we consider consolidating the conversion scripts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes the weights are same llam2 architecture, we can merge with existing script. I will try that. Thanks!
When we run on cpu and use LLam2 all weights are in torch so and we can use the same approach as phi3 seems more fault tolerant When tested on CPU from But when used Thanks |
@sineeli I think With what precision are the original pytorch checkpoints stored on disk? If they are at float16, we could just do the same (and store at float16 on disk). The disk format does not mean we need to load at that format. Anyway, as soon as you push with the comments addressed above we can merge this PR. And keep working on the actual checkpoints we ship. |
@sineeli can you make your kaggle model public? I'll pull in the script but leave the new preset off for now, we can do that on a follow up PR. |
Sure, waiting for page update. Thank! https://www.kaggle.com/models/sineeli/vicuna/keras/vicuna_1.5_7b_en |
Successfully converted checkpoints from vicuna(torch) to Keras3 compatible, please let me know if any refactoring needed.
Thanks