cross-pthread creation and destruction of WebGPU objects

[mannkdev]

Please include the following in your bug report:

Version of emscripten/emsdk:
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.28 (f11d619)
clang version 16.0.0 (https://github.com/llvm/llvm-project ea4be70cea8509520db8638bb17bcd7b5d8d60ac)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: /home/kangm/code/emsdk/upstream/bin

Failing command line in full:
If this is compile or link-time failure please include the full failing command
along with its entire output.

Full link command and output with -v appended:
Even for runtime issues it helps a lot if you can include the full link command.
Adding -v to the link command will show all of the sub-commands run which
can help us diagnose your issue.

Because the "manager" objects like WebGPU.mgrBuffer are not synchronized across pthreads/web workers, an object that's created on one thread (e.g. via wgpuDeviceCreateBuffer) cannot be destroyed (e.g. via wgpuBufferDestroy) on another, as the destroying thread cannot find the given object id in its manager map.

Is there any workaround or fix planned for this? I came across this explainer (https://gpuweb.github.io/gpuweb/explainer/#multithreading-transfer) but it seems like none of the proposed solutions are very satisfying,

[kainino0x]

Multithreading of this kind will require the underlying JS API to be multithreaded. That will happen eventually, but it will be some time before we know exactly what it looks like. Once we are making progress on it, Emscripten will be the first place we're prototyping support.

There are a few things that make this complicated, in particular synchronously sharing WebGPU objects across workers so that they can be used across threads without explicit send and async receive operations on the threads.

Vaguely, there could be a possibility of doing the proxy-to-worker thing that Emscripten for WebGL, has where it proxies all WebGL commands to a single thread so the underlying JS API doesn't have to be multithreaded. However our team isn't pursuing this right now as we are currently working on other WebGPU improvements, including improving and stabilizing webgpu.h across native and emscripten.

[sbc100]

@mannkdev if we were to implement this via proxying all WebGPU calls back to the main thread would that solve your program?

@kainino0x do you think such an approach would be viable? i.e. is the number of WebGPU calls per frame generally high enough that the cost of proxying would be prohibitive?

FWIW I think it would be fairly easy to automatically proxy all calls back the to the main thread? Indeed maybe we be doing this by default or having the calls assert when made from a pthread?

[kainino0x]

I don't think it should be more prohibitive than proxying WebGL. I do think it would be more of a stopgap before we have real multithreading in the JS API, but it's going to be a while before we reach that point. So if it turns out not to be that difficult to proxy these, I'd be all for providing it (behind some build flag).

[kainino0x]

And maybe a runtime flag, too, in wgpuCreateInstance? Sometimes applications might want to use two completely independent WebGPU instances on different threads, which JS does already support, rather than proxying both instances to one thread for no reason.

[mannkdev]

Sounds good, if this could be implemented it would allow us to proceed with our planned usage of pthreads + WebGPU. Seems less than ideal performance wise, but guess we will cross that bridge and benchmark when we get to that point.

Do you have any idea of how long it might take for you guys to implement this?

And yes, having build + runtime flags is a great idea. Only one possible concern is how much further having a runtime flag would degrade performance (if it gets checked on every operation).

[sbc100]

@mannkdev do you actually want multiple threads to be able to make WebGPU calls? Or will all of them come from the same thread?

[mannkdev]

@sbc100 Yes, we have existing code that needs to be able to make WebGPU calls from multiple threads. That's how we came across this issue - those threads are currently disabled. We want to enable them, but doing so results in crashes as some threads attempt to destroy objects that were created on other threads. To clarify, these threads are part of cross-platform code and work fine on other platforms (for example D3D).

[Mintyboi]

So if it turns out not to be that difficult to proxy these, I'd be all for providing it (behind some build flag).

@kainino0x May I check if this is on your immediate roadmap and how much effort do you think it'll take to implement this?

[kainino0x]

It is not on our roadmap. It wouldn't take a ton of work to implement with the right generators, for example it could take advantage of the code generator in Dawn which generates code from a json file dawn.json. However since I don't think our team will be working on this very soon you'd also have to become familiar with that code in order to use it.

For maintenance, we'd strongly prefer something generated, because there will still be both breaking changes and new features in the webgpu.h API.

As for how to implement it, the existing WebGL proxying code in Emscripten can be used as a model (though I'm not very familiar with it unfortunately).

[warvstar]

I'm not sure forcing all proxying back to the main thread is the best idea... For instance, we have a thread/worker dedicated to webgpu and we ensure all webgpu calls happen on that thread, and it works great for us, enabling multithreaded usage. While the JSAPI changes can help, they are not needed by us at the moment, and we have a fully multithreaded Unreal Engine 5.3 project running on webgpu. We do the same thing for our WebGL backend.

I think it should be up to the developer to make sure they are calling all webgpu calls on the same thread or to toggle on JSAPI support when its available.

[sbc100]

@warvstar so you are able to use webgpu from worker thread? I didn't realize that was currently possible. And in that case I guess #20124 break your codebase? I can certainly see the proxying being an option, but we would want to add some tests for this.

@warvstar how does your application render from a background thread? Does it use something like OFFSCREENCANVAS_SUPPORT or OFFSCREEN_FRAMEBUFFER?

[warvstar]

Correct and yes we use OFFSCREENCANVAS_SUPPORT with one small emscripten change needed to make it work

var context = canvas.getContext('webgpu');
to
var context = canvas.offscreenCanvas.getContext('webgpu');

inside library_webgpu.js (wgpuInstanceCreateSurface)

[kainino0x]

@sbc100 yes, sorry, I probably should have caught this. WebGPU can be used from a worker - what we don't have is the ability to share anything between threads. I thought the proxying would only take effect with certain build options so people would be able to control this?

@warvstar good to know the changes needed are minimal. If you know a fix that works both on the main thread and on workers, I would love to take a PR.

[sbc100]

@sbc100 yes, sorry, I probably should have caught this. WebGPU can be used from a worker - what we don't have is the ability to share anything between threads. I thought the proxying would only take effect with certain build options so people would be able to control this?

But can it be used from a worker with emscripten? I guess there are no tests for that? Would it depend on OFFSCREENCANVAS_SUPPORT? We can make it is so that the proxying only kicks in with OFFSCREENCANVAS_SUPPORT disabled perhaps? Is there any other way to use webgpu from a worker?

@warvstar good to know the changes needed are minimal. If you know a fix that works both on the main thread and on workers, I would love to take a PR.

[kainino0x]

But can it be used from a worker with emscripten? I guess there are no tests for that? Would it depend on OFFSCREENCANVAS_SUPPORT? We can make it is so that the proxying only kicks in with OFFSCREENCANVAS_SUPPORT disabled perhaps? Is there any other way to use webgpu from a worker?

It can almost certainly be used without a canvas. With a canvas it sounds like it's only slightly broken, and we want it to work.

WebGPU only uses canvases for output, so you can do lots of things in WebGPU without a canvas, especially GPGPU compute. It's also possible to use an OffscreenCanvas that isn't transferred from the main thread (but this is presumably just as broken in Emscripten as using an OffscreenCanvas from the main thread).

[sbc100]

But can it be used from a worker with emscripten? I guess there are no tests for that? Would it depend on OFFSCREENCANVAS_SUPPORT? We can make it is so that the proxying only kicks in with OFFSCREENCANVAS_SUPPORT disabled perhaps? Is there any other way to use webgpu from a worker?

It can almost certainly be used without a canvas. With a canvas it sounds like it's only slightly broken, and we want it to work.

WebGPU only uses canvases for output, so you can do lots of things in WebGPU without a canvas, especially GPGPU compute. It's also possible to use an OffscreenCanvas that isn't transferred from the main thread (but this is presumably just as broken in Emscripten as using an OffscreenCanvas from the main thread).

I see.. I that case perhaps the proxying should just opt-in?

[kainino0x]

That would make sense. At least a build-time option to turn WebGPU proxying on and off would be good. We could consider a runtime switch, either thread global (easier) or somehow per-object (harder) but it probably isn't critically needed.

[sbc100]

I think build-time is the only one that is simple for us to implement at this point.

[jspanchu]

For those who came here looking to offscreen render from a webworker with webgpu, it works from 3.1.56 onwards, thanks to #21467. All that you need is to pass -pthread and -sOFFSCREENCANVAS_SUPPORT.