How to properly write to a buffer every frame?

[nikitablack]

Hello. I came from the Vulkan World and try to map wgpu to it. Unfortunately, it doesn't map 1:1. I can't even create a mapped buffer and copy data to it. Here's how I'm creating a buffer:

let storage_buffer = device.create_buffer(&wgpu::BufferDescriptor {
    label: Some("storage buffer"),
    size: 1000,
    usage: wgpu::BufferUsage::STORAGE
           | wgpu::BufferUsage::COPY_DST
           | wgpu::BufferUsage::MAP_WRITE,
     mapped_at_creation: true,
});

And it fails with the error:

Caused by:
    In Device::create_buffer
      note: label = `storage buffer`
    `MAP` usage can only be combined with the opposite `COPY`, requested MAP_WRITE | COPY_DST | STORAGE

It's confusing. Looks like I should use another flag. But since I'm creating a buffer for writing to it I'm expecting the usage to be wgpu::BufferUsage::COPY_DST, i.e. the buffer should be a destination for my data. But ok, let me try with different flags:

let storage_buffer = device.create_buffer(&wgpu::BufferDescriptor {
    label: Some("storage buffer"),
    size: 1000,
    usage: wgpu::BufferUsage::STORAGE
           | wgpu::BufferUsage::COPY_SRC // CHANGED
           | wgpu::BufferUsage::MAP_WRITE,
     mapped_at_creation: true,
});

But it still fails:

Caused by:
    In Device::create_buffer
      note: label = `storage buffer`
    `MAP` usage can only be combined with the opposite `COPY`, requested MAP_WRITE | COPY_SRC | STORAGE

What am I doing wrong?

Also, I couldn't find any guidelines regarding per-frame buffer updates. In Vulkan it's common to have multiple buffers in flight and update one that is not in use by GPU at the moment. I can't find something similar in wgpu. The only thing I saw in tutorials is to use wgpu::Buffer::map_write which returns a future. So it will block when I'll try to copy into it? Another approach is to use wgpu::Queue::write_buffer. It is even mentioned that the method is intended to have low-performance costs. But I don't understand how - I can't write to the buffer without knowing it's not used by GPU. How does the queue know that? Does wait for the previous frame completion? But in that case, it blocks and we can't say it is "low-performance".

Basically I need to copy a big amount of data every frame and I'm searching for the fastest way. So it would be great if someone could provide best practices regarding this topic.

[kvark]

Hi! I took the liberty of moving your issue into a discussion, because I feel that this isn't an actual issue reported.

Unfortunately, it doesn't map 1:1.

WebGPU is quite close to Vulkan, but data transfers between CPU and GPU are indeed one of the main points of difference. The basic rules are:

• API allows no data races
• API does the synchronization for you

So it may be difficult to wrap a Vulkan-trained head around these. One of the consequences of these rules is: staging buffers can't be used for anything other than staging. So if you have MAP_WRITE, then it can only be combined with COPY_SRC and nothing else.

I can't even create a mapped buffer and copy data to it.

You can do this just fine. There is create_buffer_init, which is used a ton by the examples.

Also, I couldn't find any guidelines regarding per-frame buffer updates

Similarly, have a look at the examples. They do per-frame updates. You don't need to rotate GPU buffers since WebGPU does the synchronization for you. Just call write_buffer before submission to fill out the data. It's on the queue, and we have a single queue atm, so it's well aware of anything that may need the buffer. It basically schedules a copy operation on the queue, there is no blocking.

If you want to squeeze the last bit of upload perf, then the wgpu::util::StagingBelt may be for you. It gets you almost the same thing as write_buffer, but with more explicit control.

[nikitablack]

Indeed, this is not an issue. The first i did thing before asking is tried to open Discussions, strange though I didn't see the tab. Thank you for moving the question.

One of the consequences of these rules is: staging buffers can't be used for anything other than staging.

I suppose that by "staging" you mean an intermediate buffer that's purpose is to deliver the data to the gpu and be forgotten. True, that in this case it should be COPY_SRC. But I'm asking about a "normal" buffer, readable on the gpu and mappable on the host. Can I create one in wgpu? I don't want to init it upon creation, the update happens later, so create_buffer_init is not for me, I suppose. But Device::create_buffer fails for me with flags combination I posted earlier.

Just call write_buffer before submission to fill out the data.

It would be great if you could give a technical explanation of how it's done. How a buffer gets updated? Is every buffer is created mapped on host? Then how do you handle the situation when the buffer is written while it's in use on the device? Or is there maybe a hidden staging buffer for every buffer?

If you want to squeeze the last bit of upload perf, then the wgpu::util::StagingBelt may be for you.

Do you know any examples of how to use it? The documentation is pretty dry for this method.

[kvark]

But I'm asking about a "normal" buffer, readable on the gpu and mappable on the host. Can I create one in wgpu?

Not in the portable core. You can use the MAPPABLE_PRIMARY_BUFFERS feature if you want to limit the portability to native code only (i.e. cut off the Web).

It would be great if you could give a technical explanation of how it's done. How a buffer gets updated? Is every buffer is created mapped on host? Then how do you handle the situation when the buffer is written while it's in use on the device? Or is there maybe a hidden staging buffer for every buffer?

Oh, this should be straightforward. Here is the pseudo-code for write_buffer:

1. create a new temp_buffer with the user data
2. create a command encoder temp_encoder, encode the copy_buffer_to_buffer(temp_buffer, destination)
3. finish the command encoder and submit it: queue.submit([temp_encoder.finish()])

Do you know any examples of how to use it? The documentation is pretty dry for this method.

I know people are using it today, but we haven't gotten into demonstrating it in the examples. We want this, though.

[nikitablack]

i.e. cut off the Web

Wait, how? I thought that WebGPU (and wgpu) is the Web. I think I lack the knowledge of how everything works under the hood. But I thought there're multiple backends (i.e. Vulkan, DirectX, Metal, etc.), and WebGPU covers the intersection of the APIs, i.e. it takes the common parts snd wraps them in a new API. I don't know well all of the APIs but isn't mapping a buffer available in all APIs?

1. create a new temp_buffer with the user data

Does that mean that the lib allocates a new temporary buffer every time I call write_buffer? Isn't that slow? Or do you allocate it from a pool? Then how big is the pool? And what if I need to update a dozen of buffers? Anyway, using staging buffers involves more copying than writing directly to a mapped buffer. It's not clear why you chose this path, could you please elaborate more?

[kvark]

Wait, how? I thought that WebGPU (and wgpu) is the Web

WebGPU is made for the Web, but the API makes a lot of sense even outside of the web - on native only as a modern replacement for OpenGL, as a nicely standardized and fairly powerful very portable API. Checkout my Fosdem talk if you have a moment! wgpu-rs targets both the web and the native, and we have a few features that are native-only.

I don't know well all of the APIs but isn't mapping a buffer available in all APIs?

Mapping is available, but the Web as a target platform makes it much more complicated. In most browsers today, the GPU process is separated from the content process. So the mapping would have to be done not only across CPU and GPU, but also across the process boundary. This is very tricky, and the group opted into a more restricted version of the API that doesn't require this mapping. I.e. the staging buffers on the Web mostly represent IPC shared memory.

Does that mean that the lib allocates a new temporary buffer every time I call write_buffer? Isn't that slow?

I wrote pseudo-code that shows how it works conceptually. In practice we have a deque of staging buffers, internally, that we rotate as the write_xx are issued, and GPU finishes the copies. We grow the deque as necessary. The expectation is that mostly write_buffer doesn't allocate or create anything: it just grabs a piece from that staging deque ("staging belt") and uses it.

Anyway, using staging buffers involves more copying than writing directly to a mapped buffer.

Staging buffers help when the GPU is using the target buffer. If you know the GPU isn't going to be using it, or want to structure your game in a way that relies on that, you should look at the StagingBelt: it's basically user code (it layers on top of the API) that does uploads via mapping.
Also, if you are talking about initialization, then mapped_at_creation: true is your friend for minimal copies.

[nikitablack]

Checkout my Fosdem talk

I already did, thanks. Sorry if I'm asking the same questions :)

Mapping is available, but the Web as a target platform makes it much more complicated.

Ok, I got it - no memory mapping on the host with wgpu.

Staging buffers help when the GPU is using the target buffer

By "more copies" I mean that:

1. Copy data to mapped staging buffer on CPU.
2. Copy staging buffer to target buffer on GPU.
3. ? Above you said that there's a cross-process barrier. Probably there's one more copy between processes on CPU?

So yeah, much more than a single copy to mapped target buffer (I didn't check the "staging belt" yet, maybe it changes a lot). And please don't understand me wrong - I'm not complaining, just trying to understand :) And I understand that you're very constrained inside a browser sandbox.

Also, if you are talking about initialization, then mapped_at_creation: true is your friend for minimal copies.

And here I confused again - previously you told that there's no mapping at all. But this flag hints at something opposite. The documentation adds to my confusion: mapped_at_creation: Allows a buffer to be mapped immediately after they are made. It does not have to be BufferUsage::MAP_READ or BufferUsage::MAP_WRITE, all buffers are allowed to be mapped at creation. So in the end it is mapped without specifying a feature?

[kvark]

Ok, I got it - no memory mapping on the host with wgpu.

wgpu allows you to have mapping with that native-only extension. WebGPU does not.

By "more copies" I mean that:

Your assessment is correct.

And here I confused again - previously you told that there's no mapping at all. But this flag hints at something opposite. The documentation adds to my confusion: mapped_at_creation: Allows a buffer to be mapped immediately after they are made. It does not have to be BufferUsage::MAP_READ or BufferUsage::MAP_WRITE, all buffers are allowed to be mapped at creation. So in the end it is mapped without specifying a feature?

Ok, sorry for confusing you! mapped_at_creation is not really a generic mapping mechanism. It's only an efficient way to initialize something, and that's a part of WebGPU proper. Important to note that create_buffer_init is an utility function that is implemented on top of mapped_at_creation, it has no magic besides that, it's all user space.

Initialization is special because we know the GPU isn't using the resource yet, so no-data-races property is guaranteed. Therefore, WebGPU makes it available for all buffers, regardless of their usage flags.

[nikitablack]

Thanks a lot @kvark! I appreciate your help and great explanations. I don't have any more questions regarding the topic. And as a conclusion I'll summarize what you told me regarding per-frame buffer updates:

1. The fastest way would be to map buffers on the host. But this requires MAPPABLE_PRIMARY_BUFFERS feature to be presented. Atm, this is desktop only.
2. Update a buffer can be achieved with write_buffer. But this function calls to internal staging buffers management. Probably it can create new allocations. Also, updating this way involves lots of copies.
3. wgpu::util::StagingBelt is the fastest portable way as of today. The documentation states that the method allows to avoid an extra data copy.

Thank you!

P.S.: is there an active wgpu discord or slack channel?

[kvark]

Also, updating this way involves lots of copies.
wgpu::util::StagingBelt is the fastest portable way as of today.

The situation is more complex that this. When targeting the Web, the data also has to go through the WASM boundary, which requires basically accepting &[u8] on our side. This means, your can't really write even into the shared IPC buffers mapped, since you need to cross that WASM boundary first. As a result, using buffer mapping on the Web (which has to be followed by copy_buffer_to_buffer )is always going to be slower then write_buffer...

So the recommendation today is to use mapped_at_creation for initialization and write_buffer for updates. The good thing is - this is a fairly simple API (compared to buffer mapping), so it's kinda nice where the simple thing becomes recommended.

P.S.: is there an active wgpu discord or slack channel?

We have matrix channels "#wgpu:matrix.org" and `#wgpu-users:matrix.org".
Some of our members also monitor Discord rust-gamedev channel.