Replies: 21 comments 46 replies
-
|
Can't seem to get the app to transcribe just yet - something I'm doing wrong here? Thanks |
Beta Was this translation helpful? Give feedback.
-
|
ok, figured it out I think, I had to create a build folder inside the go package and drop in libwhisper.a there |
Beta Was this translation helpful? Give feedback.
-
|
Hi Chris, happy new year. So there is a "Makefile" in the
If you are placing your code in the "examples" folder you can build it with:
I hope that helps! |
Beta Was this translation helpful? Give feedback.
-
|
I wanted to use this binding in my project. Also had problem to build it with missing header and I realized the CGO directives in the source files require the header to be in Now I was able to compile it program with just |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, I've put in a PR #379 now which removes the hard-coded paths from the .go file in preference for setting the C_INCLUDE_PATH and LIBRARY_PATH variables which invoking the go toolchain, and included some documentation to describe this. I hope that helps! |
Beta Was this translation helpful? Give feedback.
-
@chrisbward I added a very basic streaming example here: https://github.com/djthorpe/go-whisper/ It creates all sorts of crashes (I need to look into thread safety, I assume it's bacause I am launching goroutines where I shouldn't). But may help you a bit more. |
Beta Was this translation helpful? Give feedback.
-
|
Hello Everybody, First of all thank you very much for all the work that was poured into this. I created my build folder with all executables. github.com/ggerganov/whisper.cpp/bindings/go
Any advice/piece of help is very much appreciated. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @mbaxamb3333 this is almost certainly because cgo cannot find the whisper.h file. It's not enough to include them at runtime, it needs to be done as part of the compilation (go build). Therefore, if you are using: Please change that to: The directory argument should be where you have |
Beta Was this translation helpful? Give feedback.
-
|
Hello, @djthorpe thank you very much for taking the time. Unfortunately, it still did not work.. |
Beta Was this translation helpful? Give feedback.
-
|
As you can notice, in the dependencies folder on the left I stored bot .h and .a files. |
Beta Was this translation helpful? Give feedback.
-
|
I must mention that, even if I try to run the package tests from the original repository, with the original path, it fails from the same reason.. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @mbaxamb3333 can you run the following and see what happens, in a fresh empty directory: I'd be interested to see if that compiles. If it doesn't work, can you send me the output? |
Beta Was this translation helpful? Give feedback.
-
|
When running the command from the makefile everything works fine. github.com/ggerganov/whisper.cpp/bindings/go/var/folders/y0/dbsx2zd55lz31wt__nmdgm840000gn/T/go-build1048629891/b084/params.cover.go:11:10: fatal error: 'whisper.h' file not found |
Beta Was this translation helpful? Give feedback.
-
|
Great. In that case, I would take a look at the Makefile in the bindings/go directory and see what it is doing differently to make it work. The most important part of the makefile is as follows: It's really just important that INCLUDE_PATH contains the file whisper.h and the LIBRARY_PATH contains the file libwhisper.a - these should be on the same line as the "go build" command. |
Beta Was this translation helpful? Give feedback.
-
|
Golang bindings are 45 times slower than the C++ binary when transcoding
C++ and Golang examples are compiled following the readme. I haven't profiled the Golang bindings yet. Raw resultsC++ binarytime ./main -m ./models/ggml-tiny.en.bin -f ./samples/jfk.wav
whisper_init_from_file: loading model from './models/ggml-tiny.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 384
whisper_model_load: n_audio_head = 6
whisper_model_load: n_audio_layer = 4
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 384
whisper_model_load: n_text_head = 6
whisper_model_load: n_text_layer = 4
whisper_model_load: n_mels = 80
whisper_model_load: f16 = 1
whisper_model_load: type = 1
whisper_model_load: mem required = 387.00 MB (+ 3.00 MB per decoder)
whisper_model_load: kv self size = 2.62 MB
whisper_model_load: kv cross size = 8.79 MB
whisper_model_load: adding 1607 extra tokens
whisper_model_load: model ctx = 73.58 MB
whisper_model_load: model size = 73.54 MB
system_info: n_threads = 4 / 16 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
main: processing './samples/jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:07.740] And so my fellow Americans ask not what your country can do for you
[00:00:07.740 --> 00:00:10.740] ask what you can do for your country
whisper_print_timings: load time = 186.77 ms
whisper_print_timings: mel time = 83.82 ms
whisper_print_timings: sample time = 16.98 ms
whisper_print_timings: encode time = 791.23 ms / 197.81 ms per layer
whisper_print_timings: decode time = 289.46 ms / 72.36 ms per layer
whisper_print_timings: total time = 1374.19 ms
real 0m1.428s
user 0m4.208s
sys 0m0.280sGolang bindingstime ./build/go-whisper -model ./models/ggml-tiny.en.bin samples/jfk.wav
whisper_init_from_file: loading model from './models/ggml-tiny.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 384
whisper_model_load: n_audio_head = 6
whisper_model_load: n_audio_layer = 4
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 384
whisper_model_load: n_text_head = 6
whisper_model_load: n_text_layer = 4
whisper_model_load: n_mels = 80
whisper_model_load: f16 = 1
whisper_model_load: type = 1
whisper_model_load: mem required = 387.00 MB (+ 3.00 MB per decoder)
whisper_model_load: kv self size = 2.62 MB
whisper_model_load: kv cross size = 8.79 MB
whisper_model_load: adding 1607 extra tokens
whisper_model_load: model ctx = 73.58 MB
whisper_model_load: model size = 73.54 MB
Loading "./samples/jfk.wav"
...processing "./samples/jfk.wav"
[ 0s-> 7.74s] And so my fellow Americans ask not what your country can do for you
[ 7.74s->10.74s] ask what you can do for your country
real 1m3.919s
user 4m7.851s
sys 0m6.771sRelated issue: #421 |
Beta Was this translation helpful? Give feedback.
-
|
Could this hard coded
|
Beta Was this translation helpful? Give feedback.
-
|
I made three samples and tested their performance on MLK's full "I have a dream" speech:
So there's a significant cost to using either of the official bindings. I'm not sure where the cost comes in on the low-level bindings, because my cgo looks extremely similar to what it's doing, but it's repeatably much faster. All programs are available here (I've linked directly to the version I used for these tests) The default example program gives 27s (!) when run with |
Beta Was this translation helpful? Give feedback.
-
|
@jaybinks on a 53 minute gil strang lecture, with the whisper-cpp main (4-thread default): 14:15 system_info from
|
Beta Was this translation helpful? Give feedback.
-
|
For kicks, I edited the bindings to remove the progress callbacks on the theory that that caused frequent calls across the go -> C boundary, and that dropped the time for the low-level bindings to 14:19 from 62:28 - pretty conclusive evidence that they are a major issue. Here's the diff: --- whisper.go 2023-06-26 08:49:04
+++ whisper.2.go 2023-06-29 15:43:00
@@ -326,10 +326,10 @@
// It seems this approach can offer some speedup in some cases.
// However, the transcription accuracy can be worse at the beginning and end of each chunk.
func (ctx *Context) Whisper_full_parallel(params Params, samples []float32, processors int, encoderBeginCallback func() bool, newSegmentCallback func(int)) error {
- registerEncoderBeginCallback(ctx, encoderBeginCallback)
- registerNewSegmentCallback(ctx, newSegmentCallback)
- defer registerEncoderBeginCallback(ctx, nil)
- defer registerNewSegmentCallback(ctx, nil)
+ // registerEncoderBeginCallback(ctx, encoderBeginCallback)
+ // registerNewSegmentCallback(ctx, newSegmentCallback)
+ // defer registerEncoderBeginCallback(ctx, nil)
+ // defer registerNewSegmentCallback(ctx, nil)
if C.whisper_full_parallel((*C.struct_whisper_context)(ctx), (C.struct_whisper_full_params)(params), (*C.float)(&samples[0]), C.int(len(samples)), C.int(processors)) == 0 {
return nil |
Beta Was this translation helpful? Give feedback.
-
Can we reuse the same model instance to process multiple wav files which are in different languages?I have 20 wav files which are in different languages and totally unrelated. I create one model using a multi-lang model, and then create a new Context using So my question is - are we supposedly to reuse one model and just create a new context for each new wav file? My understanding is that the model inference is somewhat stateful - would the first wav file / context influence the second wav file /context in some way? Btw thanks for building such a great project! |
Beta Was this translation helpful? Give feedback.
-
|
The C api does not have the concept of model. It only has |
Beta Was this translation helpful? Give feedback.
-
|
I haven't looked into the Go API in details, but regarding the C API, you should be able to achieve what you need by simply setting Line 248 in 86ef64a This will tell whisper to ignore any past transcriptions (i.e. clear the state). |
Beta Was this translation helpful? Give feedback.
-
|
I'm right that we can't do something like this for i := 0; i < 10; i++{
go func(){ // new goroutine
ctx := whisper.NewContext()
ctx.Process(...)
}()
}I'm having a data race error right now. |
Beta Was this translation helpful? Give feedback.
-
|
Any qualms against including pre-compiled static libraries for the go bindings (if not more?). The https://github.com/rogchap/v8go is a nice demonstration of how doing prevents having to worry about the c toolchain and the library can effectively be treated as a native go library. See https://github.com/rogchap/v8go/blob/master/cgo.go and https://github.com/rogchap/v8go/blob/master/.github/workflows/v8build.yml for how they build those static libraries. |
Beta Was this translation helpful? Give feedback.
-
|
I've done similar to what your saying here.
https://github.com/jaybinks/goConstmeWhisper
And yea, I kinda agree.
This allows you to simply download a binary .so of whisper without being
required to build it yourself.
That being said, it's also not super feasible since your going to get a
very unoptimized whisper library
…On Sun, 25 June 2023, 9:00 am Travis Cline, ***@***.***> wrote:
Any qualms against including pre-compiled libraries for the go bindings
(if not more?). The https://github.com/rogchap/v8go is a nice
demonstration of how doing prevents having to worry about the c toolchain
and the library can effectively be treated as a native go library.
See https://github.com/rogchap/v8go/blob/master/cgo.go and
https://github.com/rogchap/v8go/blob/master/.github/workflows/v8build.yml
for how they build those binaries.
—
Reply to this email directly, view it on GitHub
<#312 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AALQR67V4XJG7HTBBHL6GUDXM5WRBANCNFSM6AAAAAATG4MHKQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
That doesn't seem to match my definition of 'infeasible' -- can you elaborate? |
Beta Was this translation helpful? Give feedback.
-
|
It would be interesting to see how your performance observations scale with
much longer audio durations.
or, simply looping over the same audio multiple times after loading the
model etc.
My gut says the difference must be memory copies vs pointers or something
like this.
…On Thu, 29 Jun 2023 at 03:11, Bill Mill ***@***.***> wrote:
I made three samples and tested their performance on MLK's full "I have a
dream" speech:
- binding libwhisper.so directly and using cgo: 24s
- using the "low-level" bindings: 31s
- using the "high-level" bindings: 55s
So there's a significant cost to using either of the official bindings.
I'm not sure where the cost comes in on the low-level bindings, because my
cgo looks extremely similar to what it's doing, but it's repeatably much
faster.
All programs are available here
<https://github.com/llimllib/blisper/tree/abad02796b6edd51fb01a9e529a46fd4420b6d85>
(I've linked directly to the version I used for these tests)
—
Reply to this email directly, view it on GitHub
<#312 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AALQR6YIEVHHHBRGH3QUCFDXNRQVTANCNFSM6AAAAAATG4MHKQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
--
Sincerely
Jay
|
Beta Was this translation helpful? Give feedback.
-
|
I had read that the go > boundary was a choke point , but never in my
wildest dreams did I ever consider it was THIS bad.
WOW .
Just for kicks, how does your sotest / lltest compare against the native C
CLI app on the same file / system / settings ??
…On Fri, 30 Jun 2023 at 05:44, Bill Mill ***@***.***> wrote:
For kicks, I edited the bindings to remove the progress callbacks on the
theory that that caused frequent calls across the go -> C boundary, and
that dropped the time for the low-level bindings to 14:19 from 62:28 -
pretty conclusive evidence that they are a major issue.
Here's the diff:
--- whisper.go 2023-06-26 08:49:04+++ whisper.2.go 2023-06-29 15:43:00@@ -326,10 +326,10 @@
// It seems this approach can offer some speedup in some cases.
// However, the transcription accuracy can be worse at the beginning and end of each chunk.
func (ctx *Context) Whisper_full_parallel(params Params, samples []float32, processors int, encoderBeginCallback func() bool, newSegmentCallback func(int)) error {- registerEncoderBeginCallback(ctx, encoderBeginCallback)- registerNewSegmentCallback(ctx, newSegmentCallback)- defer registerEncoderBeginCallback(ctx, nil)- defer registerNewSegmentCallback(ctx, nil)+ // registerEncoderBeginCallback(ctx, encoderBeginCallback)+ // registerNewSegmentCallback(ctx, newSegmentCallback)+ // defer registerEncoderBeginCallback(ctx, nil)+ // defer registerNewSegmentCallback(ctx, nil)
if C.whisper_full_parallel((*C.struct_whisper_context)(ctx), (C.struct_whisper_full_params)(params), (*C.float)(&samples[0]), C.int(len(samples)), C.int(processors)) == 0 {
return nil
—
Reply to this email directly, view it on GitHub
<#312 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AALQR62BBVO76TAOQ4RFJ4DXNXLJZANCNFSM6AAAAAATG4MHKQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Sincerely
Jay
|
Beta Was this translation helpful? Give feedback.
-
|
Sorry I missed this message, github discussions doesn't thread correctly via email it seems, so I got no notification about it. Using |
Beta Was this translation helpful? Give feedback.
-
|
When I use I have seen the same query raised by someone. issue#597. I also tried removing Can you please tell me how I should resolve this issue as I want to bind in my own Golang package. It asks me to compile Could this error be due to the low version of my Mac system? I'm using an Intel Core i7 processor and version 12.6.3. If so, I'll update my system (it's slow to update and I want to make sure it's not the system version first) |
Beta Was this translation helpful? Give feedback.
-
|
When I run
|
Beta Was this translation helpful? Give feedback.
-
|
I'm not an expert, but it seems like that's just a warning and the build is working |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the reply, but |
Beta Was this translation helpful? Give feedback.
-
|
The execution of Then in the parent directory, it should generate a target file called |
Beta Was this translation helpful? Give feedback.
-
|
Could you please tell me how to use flags?
`Loading "--language" system_info: n_threads = 8 / 8 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | METAL = 0 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | COREML = 0 | OPENVINO = 0 | Loading "ru" |
Beta Was this translation helpful? Give feedback.
-
|
Try
|
Beta Was this translation helpful? Give feedback.
-
|
Hi, I am trying to execute |
Beta Was this translation helpful? Give feedback.
-
|
I'm getting the same error in different scenario, I used the bindings and I'm running this command: LIBRARY_PATH=./whisper.cpp C_INCLUDE_PATH=./whisper.cpp go build .And I get this output: /opt/homebrew/Cellar/go/1.21.0/libexec/pkg/tool/darwin_arm64/link: running cc failed: exit status 1
ld: Undefined symbols:
_MTLCopyAllDevices, referenced from:
_ggml_metal_init in libwhisper.a[7](ggml-metal.o)
_MTLCreateSystemDefaultDevice, referenced from:
_ggml_backend_metal_buffer_type_alloc_buffer in libwhisper.a[7](ggml-metal.o)
_ggml_backend_metal_buffer_type_get_max_size in libwhisper.a[7](ggml-metal.o)
_ggml_backend_metal_buffer_from_ptr in libwhisper.a[7](ggml-metal.o)
_ggml_metal_init in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_MTLCaptureDescriptor, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_MTLCaptureManager, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_MTLCompileOptions, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_MTLComputePassDescriptor, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_NSBundle, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_NSProcessInfo, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_NSString, referenced from:
in libwhisper.a[7](ggml-metal.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [build] Error 1 |
Beta Was this translation helpful? Give feedback.
-
|
#1899 Related issue. |
Beta Was this translation helpful? Give feedback.
-
|
I've created a PR to setup Makefile env vars properly and got my binding running. #1530 |
Beta Was this translation helpful? Give feedback.
-
|
Has anyone had success building the whisper.cpp library using CUBLAS, OPENBLAS, etc and then using that from a go app? When I use I opened #1553 to ask about this but thought I should drop a comment here as well. |
Beta Was this translation helpful? Give feedback.
-
|
Hi , does someone has similar issue? I got it run as a container in Kubernetes cluster (intel CPU) but it's very slow. I suspect no hardware acceleration enabled but not sure how to do that. below is my dockerfile. Thanks |
Beta Was this translation helpful? Give feedback.
-
|
Hey, that's a different thing - and very related to your OS and docker setup, not with metal and mac os + arm. In any case look at docker capabilities to enable gpu access - that's very dependent on drivers and so on: https://docs.docker.com/compose/gpu-support/ |
Beta Was this translation helpful? Give feedback.
-
|
no GPU, just Intel CPU for the inference. I suspect the AVX or F16C capacity is not leveraged by the golang binary. |
Beta Was this translation helpful? Give feedback.
-
|
from the log, it looks like the AVX is already enabled when building libwhisper.a |
Beta Was this translation helpful? Give feedback.
-
|
I have some questions about this golang binding implementation. I made it so far, that I can use the if on the demo files that come with it. Also one other wav-file gets analysed properly, another wav-file is getting analysed as "Musik" :D even tho it i a normal speech. So in general I have some questions:
Does it benefit the tools performance, if this is getting run on an Intel CPU with an iGPU? Thanks in advance! |
Beta Was this translation helpful? Give feedback.
-
|
It would also be awesome if it could take all common audio codecs/formats (AC3, DTS, DTS-HD MA, TrueHD, TrueHD Atmos, OGG/OGA, MP4/M4A, ASF/WMA, MP3, FLAC, ALAC, DST, OPUS) of another process, that would be able to take in anything and convert it to WAV, so it can process it :) At least it would be cool on the long run. So this can be used for analysing every Video/Playbook and eighter create subtitles, or just to index the words it contains. |
Beta Was this translation helpful? Give feedback.
-
|
I'm not the author but based on my usage, live streaming would be possible if you buffered chunks (as in each 10s streaming you run the model on a goroutine). Converting from stream to chunks would be on your own. I think that even Context.Process taking a full buffer and its segment/progress callbacks can be wrapped with an io.Reader. |
Beta Was this translation helpful? Give feedback.
-
|
Question: C++/C noobie here. Can we also expose the basic stream implementation in the bindings? Or is that not possible for some reason? I saw djthorpe has his own Go implementation that's a bit buggy and hard to get running relative to this. Wondering if we can do without another implementation and only maintain this one stream here! |
Beta Was this translation helpful? Give feedback.
-
|
Hello trying to follow the Readme but getting this error when I run Any ideas? |
Beta Was this translation helpful? Give feedback.
-
This didn't help, unfortunately. |
Beta Was this translation helpful? Give feedback.
-
|
Sorry to hear that. I do have an example of buiding with CUDA enabled libggml / libwhisper / go bindings in my project here, if that's at all useful? https://github.com/mutablelogic/go-whisper I use pkgconfig to bind to all the libraries needed and I start by generating all the pkgconfig files first. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks David, appreciate your taking the time to reply. Will check out your project to see if I can get some pointers. |
Beta Was this translation helpful? Give feedback.
-
|
Hey! If anyone is still interested: |
Beta Was this translation helpful? Give feedback.
-
|
Definitely still interested! Thank you very much! Will take a look. |
Beta Was this translation helpful? Give feedback.
-
|
It seems that go bindings cannot be build using apple clang, but if I use brew version of llvm, I get errors as it is not able to use apple frameworks named in bindings. its M1 mac on macOS 14.4 |
Beta Was this translation helpful? Give feedback.
and others
-
General discussion about the
gobindings forwhisper.cppCurrent state: bindings/go
Beta Was this translation helpful? Give feedback.
All reactions