[Web] BiRefNet_T not working on webgpu

[guschmue]

Describe the issue

https://huggingface.co/onnx-community/BiRefNet_T does not work on webgpu

To reproduce

See https://huggingface.co/onnx-community/BiRefNet_T

Urgency

No response

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.19

Execution Provider

'webgpu' (WebGPU)

[guschmue]

I looked briefly at this: the model takes a lot of memory for activations and will not work with wasm32.
If I run it on webgpu I see it using close to 7GB gpu memory. In theory if your gpu has that kind of memory it should work BUT the model uses GatherND and ScatterND which we have not implemented for webgpu and falls back to wasm just to run out of memory on the wasm end.
We could implement GatherND and ScatterND what should make it work on high end gpu's but most won't have those high end gpu's.

[tidus2102]

hi @guschmue, thanks for your support. It's great if you can make it works on webgpu (at least with high-end gpus). About the memory, you means we can only run this model with wasm64 right?

[fs-eire]

If we can make GatherND and ScatterND working on WebGPU, by using external data side-loading it is possible to make it work in wasm32. And there is an ongoing effort to support wasm64 build.

[xenova]

Possibly related: https://huggingface.co/onnx-community/DepthPro-ONNX also throws an error for me:

An uncaught WebGPU validation error was raised: The number of storage buffers (36) in the Compute stage exceeds the maximum per-stage limit (8).

• While validating binding counts
• While validating [BindGroupLayoutDescriptor]
• While calling [Device].CreateComputePipeline([ComputePipelineDescriptor "Concat"]).

Another error:

Uncaught (in promise) Error: [WebGPU] Kernel "[Concat] /encoder/Concat_23" failed. Error: non concat dimensions must match

---

The model works correctly in Node.js (CPU)

[xenova]

https://huggingface.co/briaai/RMBG-2.0 is a new birefnet-based model for state-of-the-art background removal. Would be useful to test too!

[tidus2102]

Hi, just a kindly check if there have been any updates or progress regarding this issue? Thank you

[fs-eire]

The The number of storage buffers exceeds the maximum per-stage limit. is a known issue for Concat. this issue is being tracked.

[tidus2102]

If we can make GatherND and ScatterND working on WebGPU, by using external data side-loading it is possible to make it work in wasm32. And there is an ongoing effort to support wasm64 build.

hi, how about this?

[fs-eire]

#22847 and #22755 introduced implementation of GatherND and ScatterND. Please allow one day or 2 for the pipeline to publish a nightly package.