esbuild wasm vs binary

[LarsDenBakker]

Thanks a lot for esbuild. Im going to ship esbuild as part of the test runner and dev server at https://github.com/modernweb-dev/web. One thing in still wondering about is whether to use the esbuild wasm build or the binary.

The docs mention wasm as a fallback, but it seems to me like its the most reliable option to use for a node based project. The postinstall script for the binary build could be problematic for people, and may not work in a CI or restricted environments. What do you think about this? Are there any performance differences to consider?

[evanw]

Are there any performance differences to consider?

Yes. The WebAssembly version has consistently been ~10x slower than the binary version in my testing, for both the build and transform API calls. This has only been tested on my 6-core macOS laptop. I'm sure there are many reasons for this performance difference. I haven't investigated but I'd assume those reasons include:

• WebAssembly is single-threaded while native is multi-threaded
• WebAssembly is not an efficient format to represent Go code in because of how Go's coroutine implementation works
• Go needs to run a garbage collector but WebAssembly isn't designed for garbage collection

I'm also not sure how much effort has gone into the WebAssembly backend but it could potentially also have not received as much optimization work as the other native backends.

I really should add the WebAssembly version to my performance benchmarks because this question keeps coming up.

[LarsDenBakker]

Thanks for the explanation, I forgot that WASM doesn't allow multi threading yet.

I only use the single file transform, I wonder if it still creates multiple threads if I transform multiple files in parallel from JS. For either the Go or WASM implementation.

[evanw]

The Go code uses one goroutine for each API call, so each API call should be on a separate thread in the native implementation. The WebAssembly implementation still uses multiple goroutines but only has one thread available, so the API calls are still all run on a single thread.

Here's the benchmark code I made for this issue:

const fs = require('fs')
const code = fs.readFileSync('./node_modules/esbuild-wasm/lib/browser.js', 'utf8')

async function benchmark(esbuild) {
  const service = await esbuild.startService()
  for (let i = 0; i < 10; i++) await service.transform(code, {})
  await new Promise(r => setTimeout(r, 250))
  let start = Date.now()
  let promises = []
  for (let i = 0; i < 1000; i++) promises.push(service.transform(code, {}))
  await Promise.all(promises)
  let time = Date.now() - start
  service.stop()
  return time
}

async function main() {
  let native = await benchmark(require('esbuild'))
  console.log(`native: ${native}ms`)
  let wasm = await benchmark(require('esbuild-wasm'))
  console.log(`wasm: ${wasm}ms`)
  console.log(`native is ${(wasm / native).toFixed(1)}x faster than wasm`)
}

main().catch(e => setTimeout(() => { throw e }))

It even does some warmup API calls before running the benchmark to give node's WebAssembly JIT some time to do its thing, so this should be a fair comparison. Here's what I get when I run this:

native: 557ms
wasm: 7559ms
native is 13.6x faster than wasm

[LarsDenBakker]

Similar results here on my macbook pro:

native: 608ms
wasm: 6248ms
native is 10.3x faster than wasm

native: 485ms
wasm: 5797ms
native is 12.0x faster than wasm

native: 486ms
wasm: 5691ms
native is 11.7x faster than wasm

native: 462ms
wasm: 5690ms
native is 12.3x faster than wasm

I think I will see if the postinstall is giving people any issues, maybe allow opting into the wasm implementation with a flag instead.

[evanw]

That approach sounds reasonable. Please let me know if people encounter postinstall issues.

Another interesting data point: you can remove the multi-threading advantage of native by prefixing the benchmark command with GOMAXPROCS=1. Single-threaded native appears to be 3.5x faster than WebAssembly:

native: 2184ms
wasm: 7626ms
native is 3.5x faster than wasm

[evanw]

I'm going to close this since this is a question, not an issue with esbuild, and I believe that this discussion accurately answers the question.