Nodejs Addon blocking main thread. Implemented Napi::AsyncWorker #642
Conversation
|
Wonderful work! I assume this is not available yet officially in npm registry? How would I install / utilize this today if I wanted to incorporate into my Electron app (for prototype / testing)? Would I have to clone this repo, cd to this examples addon.node folder, run npm install, build, and then copy the build from their into my Electron app? I am somewhat of a newbie when it comes to node addons and trying to install non released npm packages so appreciate your patience! ;) |
|
Also curious, does the "stream" (live transcription) work with the node addon? cc @ggerganov please let me know for future where you would like me to ask these kinds of questions :) |
|
This change seems to break the Node Addon CI: https://github.com/ggerganov/whisper.cpp/actions/runs/4494126958/jobs/7906215752 Any idea what could be wrong? @GitMurf |
Hi @ggerganov i think we need to update the test. Now we use a whisper version based on promises, so we need to update the test, the actual one is using the old way. Going to create a pull request. |
|
Fixed @ggerganov on #686 |
@GitMurf Thanks! Yeah, you need to install de dependencies, build the addon and copy the final addon present on Release to the place you want to use , also you need the models too. For example im using the addon in prod moving a copy of it inside the dist folder and the model to use. |
Thanks! I got it to work! Any plans to add the streaming live transcription to the addon? If you are too busy right now, I am happy to take a crack at it if you think it is possible and pointed me in the right direction? :) |
|
Also one other question @LucasZNK ... if I package the addon with my electron app, will the user still need to manually install ffmpeg on their machine to run this addon? If so, any suggestion on a good UX for the user? I do not want to have to explain to them how to download and install ffmeg ;) is there a way to package it with node? |
…rganov#642) * fixed blocking code on node addon * modify the example to run async * format * added logic to see the whisper output * added logic to see the whisper output * removed extra function for more clean example
…rganov#642) * fixed blocking code on node addon * modify the example to run async * format * added logic to see the whisper output * added logic to see the whisper output * removed extra function for more clean example
…rganov#642) * fixed blocking code on node addon * modify the example to run async * format * added logic to see the whisper output * added logic to see the whisper output * removed extra function for more clean example
…rganov#642) * fixed blocking code on node addon * modify the example to run async * format * added logic to see the whisper output * added logic to see the whisper output * removed extra function for more clean example
…rganov#642) * fixed blocking code on node addon * modify the example to run async * format * added logic to see the whisper output * added logic to see the whisper output * removed extra function for more clean example
This pull request aims to modify the code for the Whisper Node.js addon to avoid running in a blocking manner. Previously, even though you could import it in Node.js and use a queuing system to prevent blocking the main thread, the Whisper addon would still block the entire execution of Node.js, preventing any other task from running until the transcription was complete.
For example, if you use a queue package like Bull and expect to receive three requests and execute three transcriptions in parallel, the current version of the Whisper addon would only process one request and block the main thread until the transcription is finished.
I have implemented a fix in my personal project, which allows me to execute numerous Whisper tasks simultaneously while receiving and registering new jobs to execute when threads become available. This modification will greatly enhance the efficiency and usability of the Whisper Node.js addon.
Now you can import the addon and invoke on different jobs like this: