Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversion script modification for Gemma 2 models #1691

Open
sparsh35 opened this issue Jul 11, 2024 · 1 comment
Open

Conversion script modification for Gemma 2 models #1691

sparsh35 opened this issue Jul 11, 2024 · 1 comment
Assignees
Labels
Gemma Gemma model specific issues help wanted Extra attention is needed stat:contributions welcome Add this label to feature request issues so they are separated out from bug reporting issues

Comments

@sparsh35
Copy link

Is your feature request related to a problem? Please describe.
I am trying to convert a keras Gemma model after training to HF trasnformers , so that it can be used in VLLM , for faster inference. There is a script which can convert Gemma model to transformers at tools/gemma/export_gemma_to_hf.py but it is outdated.

Describe the solution you'd like
If someone can modify it for Gemma2 models. And also write a script for LLAMA 3 models as I suppose these two are majorly used, and there is not a faster continous batching alternative in keras nlp ecosystem.

Describe alternatives you've considered

Additional context

Here is the script https://github.com/keras-team/keras-nlp/blob/master/tools/gemma/export_gemma_to_hf.py

@github-actions github-actions bot added the Gemma Gemma model specific issues label Jul 11, 2024
@mattdangerw mattdangerw self-assigned this Jul 18, 2024
@mattdangerw mattdangerw added help wanted Extra attention is needed and removed keras-team-review-pending labels Jul 18, 2024
@mattdangerw
Copy link
Member

Thanks! Agreed on both counts.

If you'd like to go in an update the script, you are more then welcome. If not, we will get to this soon.

@mattdangerw mattdangerw added the stat:contributions welcome Add this label to feature request issues so they are separated out from bug reporting issues label Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Gemma Gemma model specific issues help wanted Extra attention is needed stat:contributions welcome Add this label to feature request issues so they are separated out from bug reporting issues
Projects
None yet
Development

No branches or pull requests

3 participants