-
Notifications
You must be signed in to change notification settings - Fork 217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Whisperx Optimization #2258
base: master
Are you sure you want to change the base?
Whisperx Optimization #2258
Conversation
This must be used with our fork of the whisperx, as I had to change the library to accept more than one file and to do not use ffmpeg, We already convert the to wav before sending it to be transcribed. |
Thank you very very much @hauck-jvsh! I'll run a basic accuracy test soon, to be sure it wasn't affected by the changes. Have you pushed the changes to WhisperX to some branch in our fork? To let others know, with the batch inference approach suggested on #1539 (processing up to 16 audios at the same time) and avoiding a duplicated ffmpeg run into the library, @hauck-jvsh was able to speed up WhisperX inference on a big batch of audios with different durations up to 5x-6x on RTX 3090! |
Yes, I have pushed the changes to a branch called multi-audio |
I forgot to run the tests with all the changes disabled to compare. I ran tests only against the wav2vec model and I manage to the get whisperX only 1.6 times slower than the wav2vec. |
Default WhisperX is about 9x times slower than Wav2Vec2 large model on RTX 3090 with my test data sets, so 9 / 1.6 = 5.6x speed up :-) I've just computed the WER numbers for 2 relevant data sets:
So accuracy seems fine! |
Closes #1539