Package STT wasm for npm publishing

[ccoreilly]

No description provided.

[CLAassistant]

All committers have signed the CLA.

[Dexterp37]

Did you test the packaged library to see if it's possible to use it from within an NPM-based project?

[ccoreilly]

I did test it with an example I would like to contribute eventually to stt-examples but I think it is a good idea to test it in the CI pipeline as well. I'll add a test step for it.

[Dexterp37]

Would you kindly add a comment here explaining why, even though we have SINGLE_FILE here, it's producing multiple files? (because of the "pthread" story you mentioned, right?)

[ccoreilly]

SINGLE_FILE exists since before pthread support and it currently only inlines the wasm bytecode as a base64 string inside the main JS. Pthreads are implemented as separate (smaller) workers which actually load the main JS from within. There are some requests to inline or merge both files when using SINGLE_FILE (see emscripten#9796 and emscripten#14329)

Generally, as noted in the references above, using SINGLE_FILE generates a large asset (larger than the sum of both stt_wasm.js and stt_wasm.wasm), which means longer download and startup times. Blinded by the original goal of making it available via CDN, I thought it would be necessary to inline the wasm binary for ease of use as a library, but I just tested it and webpack can handle the separate asset perfectly (rollup should also be fine; I am not sure about parcel, but I guess it'd also work.)

I will thus remove this flag. At this point we could consider dropping es5 support (ES6 is necessary for bundlers, see emscripten#14135) but ES6 imports are still not supported in workers under Firefox (reference) so I would still add it to the github releases.

[Dexterp37]

Awesome, I agree with what you outlined. Thank you for the thorough explaination !

[reuben]

Thanks for the PR! I have a couple of comments inline, and would like to get answers to the questions by @Dexterp37 as well.

[reuben]

$ ls -lh package/dist
total 320
-r-xr-xr-x@ 1 reuben  staff   155K Oct 26  1985 stt_wasm.js
-r-xr-xr-x@ 1 reuben  staff   2.9K Oct 26  1985 stt_wasm.worker.js
$ ls -lh libstt
total 6920
-rw-r--r--@ 1 reuben  staff    26K Jul 23 14:37 COPYING
-r-xr-xr-x@ 1 reuben  staff   155K Jul 23 14:50 stt_wasm.js
-r-xr-xr-x@ 1 reuben  staff   3.2M Jul 23 14:50 stt_wasm.wasm
-r-xr-xr-x@ 1 reuben  staff   3.0K Jul 23 14:50 stt_wasm.worker.js

Is the ES6 package supposed to be so much smaller than the other one?

[ccoreilly]

No, I forgot to push a change to the Makefile to include the wasm binary. I can do it later.

[ccoreilly]

I added a test step for the wasm npm module but struggled with testing the library itself (it currently only tests that it is installed). I am only familiar with jest, which runs tests inside node and thus ES6 modules need to be transpiled to CommonJS (I tried unsuccessfully to run the ES6 modules with the experimental --experimental-vm-modules flag).
Unfortunately, there must be a bug in babel because it adds a space between ! and -- in the following line of the stt_wasm.js:

L1582               if (! --numWorkersToLoad) removeRunDependency("wasm-instantiate");

which breaks the module and does not manage to load successfully. That is why I had to skip the test that actually tests that the library loads and can be used.

I also tried the jsdom environment in jest, which emulates a browser but inside stt_wasm.js the code checks if it is running in node or the browser and it concludes (correctly) that it is running inside node but the emulated environment does not have the expected variables such as __dirname so it crashes.

Maybe someone else can extend the tests in the future (maybe it should be tested with something such as cypress?). As I wrote, for now it only tests that the library is installed and can be imported.

[reuben]

Opened a clone of this PR here to test my fixes.