Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llama : add Refact support #3061

Closed
ggerganov opened this issue Sep 7, 2023 · 2 comments · Fixed by #3329
Closed

llama : add Refact support #3061

ggerganov opened this issue Sep 7, 2023 · 2 comments · Fixed by #3329
Labels
good first issue Good for newcomers help wanted Extra attention is needed model Model specific

Comments

@ggerganov
Copy link
Owner

This is a new 1.6B code model: https://huggingface.co/smallcloudai/Refact-1_6B-fim

We should look into adding support into llama.cpp similar as to how we did for Falcon and Baichuan:

I haven't looked into the architecture yet, but I'm hoping it is similar to GPT, which we already know how to handle OK thanks to the Falcon experience. I see it also uses Alibi which we should have support for, but we haven't tested extensively, so there might be issues there.

@ggerganov ggerganov added help wanted Extra attention is needed good first issue Good for newcomers model Model specific labels Sep 7, 2023
@ggerganov ggerganov moved this to Todo in ggml : roadmap Sep 7, 2023
@Jipok
Copy link

Jipok commented Sep 9, 2023

Related: #3013

@discobot
Copy link

Let me attempt this :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers help wanted Extra attention is needed model Model specific
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants