Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Alpaca support to talk-llama example #668

Merged
merged 1 commit into from
Mar 29, 2023
Merged

Conversation

ejones
Copy link
Contributor

@ejones ejones commented Mar 28, 2023

I made a couple edits locally to get talk-llama working with alpaca-13b, and thought I'd contribute back if that's helpful.

The invocation I used is:

./talk-llama -mw ./models/ggml-base.en.bin -ml path/to/ggml-alpaca-13b-q4.bin -p Evan -t 8 --n-parts-llama 1 --verbose-prompt --prompt-file examples/talk-llama/prompts/talk-alpaca.txt

This basically boils down to making the prompt and n_parts configurable in the CLI as well as adding a sample prompt in the Alpaca format. Seems to work well with alpaca-13b; I don't have any other models to test it with. Also I admit that making the prompt configurable here exposes the {N}-style interpolation that talk-llama.cpp does, which may warrant more thought.

@ggerganov ggerganov merged commit a47e812 into ggerganov:master Mar 29, 2023
@JKeddo95
Copy link

JKeddo95 commented Mar 31, 2023

Have you seen the llama_model_load: invalid model file '../llama.cpp/models/Alpaca/13B/ggml-alpaca-13B-q4.bin' (bad magic) message? I am able to run fine in llama.cpp, but can't get it started from whisper. Can you confirm you are running newest Alpaca.cpp? Which converter (if any) did you use for your models before running?

@jmtatsch
Copy link

jmtatsch commented Apr 6, 2023

I would very much like to talk to my vicuna but am suffering from the same bad magic error.
Is the ggml format maybe too new for the llama.cpp contained in whisper.cpp?

anandijain pushed a commit to anandijain/whisper.cpp that referenced this pull request Apr 28, 2023
jacobwu-b pushed a commit to jacobwu-b/Transcriptify-by-whisper.cpp that referenced this pull request Oct 24, 2023
jacobwu-b pushed a commit to jacobwu-b/Transcriptify-by-whisper.cpp that referenced this pull request Oct 24, 2023
landtanin pushed a commit to landtanin/whisper.cpp that referenced this pull request Dec 16, 2023
iThalay pushed a commit to iThalay/whisper.cpp that referenced this pull request Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants