Vulkan compute shader dispatch latency

[mikoro]

I am porting a CUDA app to Vulkan using bgfx. I noticed (using Nsight Graphics) that there are long gaps between dispatched compute kernels. It seems that vkQueueWaitIdle is called between every compute dispatch. I am not that familiar with Vulkan but is vkQueueWaitIdle similar to cudaStreamSynchronize? In CUDA, you call cudaStreamSynchronize once at the end after dispatching multiple kernels into the stream. Could Vulkan (and bgfx) work in the same way?

[pezcode]

The Vulkan synchronization is currently suboptimal, since (as you noticed) each dispatch incurs a host wait via vkQueueWaitIdle. A better way to do this would be through pipeline barriers or events. I'm not too familiar with CUDA streams, but it might be worth noting that compute dispatches in bgfx can't be run in parallel (if that was your intention) since their order is expected to be sequential.

[mikoro]

Ok, thanks. No need to run anything in parallel, single-threaded stuff would be just fine. What I would like is something like this:

(same thread)
bgfx::dispatch(...); // async
bgfx::dispatch(...); // async
...
bgfx::dispatch(...); // async
bgfx::frame(); // wait for all compute to finish in given order

Any hints on what to change inside the renderer_vk.cpp to get something like this? I don't need any draw calls at all, everything is 100% compute. I tried commenting out some vkQueueWaitIdle calls but of course, that does not work.

[pezcode]

I suppose you'd have to make changes here:

bgfx/src/renderer_vk.cpp

Line 6662 in 7410288

VK_CHECK(vkEndCommandBuffer(m_commandBuffer) );

Currently each dispatch ends and submits the command buffer (I guess stream in cuda terms), followed by vkQueueWaitIdle. It might be as simple as making the code between vkEndCommandBuffer and vkBeginCommandBuffer conditional on !isCompute. My understanding of all the internals is rather poor, however, so it could be more involved.

[pezcode]

Quick follow-up: #2402 reworked a bit of code around this, removing unnecessary CPU stalls. If you want your compute dispatches to run in parallel on the GPU (if you make sure there are no memory hazards), you can comment out this if block:

bgfx/src/renderer_vk.cpp

Line 6744 in 0afe0d2

if (wasCompute && isCompute)

Whether Vulkan actually runs the compute dispatches in parallel is another question.