Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Backport the performance improvement from llama.cpp #709

Closed
kovaacs opened this issue Apr 1, 2023 · 2 comments
Closed

Backport the performance improvement from llama.cpp #709

kovaacs opened this issue Apr 1, 2023 · 2 comments
Labels
performance CPU and memory usage - results and comparisons

Comments

@kovaacs
Copy link

kovaacs commented Apr 1, 2023

It would be very cool if the performance improvements from ggerganov/llama.cpp#613 could be backported to this repo.

I couldn't find an issue for this, if there is one, I'm happy to close this.
@edwios
Copy link

edwios commented Apr 2, 2023 •
edited
Loading

Related. #702

Since whisper.cpp shares the same llama.* and ggml.* files with llama.cpp, I think it is just one step away - convert the whisper model from ggml into the new ggjt format.

@ggerganov
Copy link
Owner

Just updated ggml and the performance should be much better.
I observe > x2 overall improvement on Apple Silicon

No need to generate new models - everything should just work

@ggerganov ggerganov added the performance CPU and memory usage - results and comparisons label Apr 10, 2023
darth-vader-lg added a commit to darth-vader-lg/whisper.cpp that referenced this issue Apr 16, 2023
@darth-vader-lg
Revert "ggml : backport llama.cpp updates (close ggerganov#709)" to r…
4726f97 
…estore the speed

This reverts commit 69b8503.
jacobwu-b pushed a commit to jacobwu-b/Transcriptify-by-whisper.cpp that referenced this issue Oct 24, 2023
@ggerganov
ggml : backport llama.cpp updates (close ggerganov#709)
301274b 
- About x2 overall performance improvement on Apple Silicon
- Results should now be the same for different number of threads (not
  tested)
jacobwu-b pushed a commit to jacobwu-b/Transcriptify-by-whisper.cpp that referenced this issue Oct 24, 2023
@ggerganov
ggml : backport llama.cpp updates (close ggerganov#709)
02b6b47 
- About x2 overall performance improvement on Apple Silicon
- Results should now be the same for different number of threads (not
  tested)
landtanin pushed a commit to landtanin/whisper.cpp that referenced this issue Dec 16, 2023
@ggerganov
ggml : backport llama.cpp updates (close ggerganov#709)
549af69 
- About x2 overall performance improvement on Apple Silicon
- Results should now be the same for different number of threads (not
  tested)
iThalay pushed a commit to iThalay/whisper.cpp that referenced this issue Sep 23, 2024
@ggerganov @iThalay
ggml : backport llama.cpp updates (close ggerganov#709)
6d6482c 
- About x2 overall performance improvement on Apple Silicon
- Results should now be the same for different number of threads (not
  tested)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance CPU and memory usage - results and comparisons
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@edwios @ggerganov @kovaacs