DX12: One frame delay of platform window size update

[devkaiwang]

Version Info:

Dear ImGui 1.90.0 (19000)
--------------------------------
sizeof(size_t): 8, sizeof(ImDrawIdx): 2, sizeof(ImDrawVert): 20
define: __cplusplus=199711
define: _WIN32
define: _WIN64
define: _MSC_VER=1938
define: _MSVC_LANG=201402
define: IMGUI_HAS_VIEWPORT
define: IMGUI_HAS_DOCK
--------------------------------
io.BackendPlatformName: imgui_impl_win32
io.BackendRendererName: imgui_impl_dx12
io.ConfigFlags: 0x00000443
 NavEnableKeyboard
 NavEnableGamepad
 DockingEnable
 ViewportsEnable
io.ConfigViewportsNoDecoration
io.ConfigInputTextCursorBlink
io.ConfigWindowsResizeFromEdges
io.ConfigMemoryCompactTimer = 60.0
io.BackendFlags: 0x00001C0E
 HasMouseCursors
 HasSetMousePos
 PlatformHasViewports
 HasMouseHoveredViewport
 RendererHasVtxOffset
 RendererHasViewports
--------------------------------
io.Fonts: 1 fonts, Flags: 0x00000000, TexSize: 512,64
io.DisplaySize: 1264.00,761.00
io.DisplayFramebufferScale: 1.00,1.00
--------------------------------
style.WindowPadding: 8.00,8.00
style.WindowBorderSize: 1.00
style.FramePadding: 4.00,3.00
style.FrameRounding: 0.00
style.FrameBorderSize: 0.00
style.ItemSpacing: 8.00,4.00
style.ItemInnerSpacing: 4.00,4.00

My Issue/Question:

When we dragging and resizing a dock window (Platform Window), there is one frame delay of the update of the window size for dock window.

I put logs in these places to show the order:

• At the beginning of each new frame in main loop. So we know when a new frame of ImGui begins.
• In function ImGui_ImplDX12_SetWindowSize where framebuffers recreated due to new size.
• In ImGui_ImplWin32_WndProcHandler_PlatformWindow where WM_SIZE event got handled.

Here is the log and I added extra comments using //. (293,692) is the size of the window when resizing.

===> New ImGui Frame.  // Frame N
WM_SIZE event of platform window: (293,692). // WM_SIZE happens in Frame N
ImGui_ImplDX12_SetWindowSize: (293, 692).  // Backbuffer recreated in Frame N

===> New ImGui Frame. // Frame N+1
UpdateViewportsNewFrame: (293,692). // Platform size info got updated in Frame N+1

===> New ImGui Frame. // Frame N+1
//...

In fact, as the order is incorrect, PlatformRequestResize flag is cleared in ImGuiViewportP::ClearRequestFlags() before it can actually be used to trigger the resize work.

Screenshots/Video
By adding a sleep in the main loop to make the refresh rate low, issue can be seen easily.

//...
::Sleep(80); // sleep for 80 milliseconds.

OutputDebugStringA("===> New ImGui Frame.\n");

// Start the Dear ImGui frame
ImGui_ImplDX12_NewFrame();
ImGui_ImplWin32_NewFrame();
ImGui::NewFrame();
//...

Screen record:

platform_window_resizing.mp4

[devkaiwang]

After changing swapchain scaling from DXGI_SCALING_STRETCH to DXGI_SCALING_NONE. I could not see resizing glitch and zoom-in/out effact when dragging the border of the window. I think this change could be a workaround.

imgui/backends/imgui_impl_dx12.cpp

Line 902 in 8add6bc

sd1.Scaling = DXGI_SCALING_STRETCH;

[devkaiwang]

I also logged for the DX9 and DX11 backends they have the same order issue. Since they don't have scaling-stretch option for swapchain, so I did not notice the issue. So I think it should be a common issue of docking (on Windows?).

[ocornut]

In fact, as the order is incorrect, PlatformRequestResize flag is cleared in ImGuiViewportP::ClearRequestFlags() before it can actually be used to trigger the resize work.

This is intentional. PlatformRequestResize is used when resizing from Platform/OS decorations (e.g. window resizing border) which are disabled by default as io.ConfigViewportsNoDecoration == true.
I will add comments near that ClearRequestFlags() to clarify this.

Resizing Platform window from ImGui displayed decorations

• 1. Usually starts from Begin()->UpdateWindowManualResize() which modifies window->Size.
• 2. Begin()->if (window->ViewportOwned) does viewport->Size = window->Size.
• 3. After EndFrame(), UpdatePlatformWindows() will call g.PlatformIO.Platform_SetWindowSize(viewport, viewport->Size); AND g.PlatformIO.Renderer_SetWindowSize(viewport, viewport->Size);. The Platform_SetWindowSize() will lead to a WM_SIZE setting PlatformRequestResize = true which is immediately cleared at the end of `UpdatePlatformWindows().

Resizing Platform window from Platform/OS displayed decorations

• 1. WndProc handler receive WM_SIZE, set viewport->PlatformRequestResize = true
• 2. NewFrame()->UpdateViewportsNewFrame() goes in if (viewport->PlatformRequestResize) block, calls g.PlatformIO.Platform_GetWindowSize(viewport) and store in viewport->Size. PlatformRequestResize is cleared at end of function.
• 3. After EndFrame(), UpdatePlatformWindows() will call g.PlatformIO.Renderer_SetWindowSize(viewport, viewport->Size)

I believe your log is partially incorrect (some incorrect recording or manipulation or missing details) ?

By adding a sleep in the main loop to make the refresh rate low, issue can be seen easily.

If I add a Sleep(80) or larger value e.g. Sleep(200) you can notice that the error only happens for very short while (at least on my setup), not for 200 ms. So it doesn't make it any "easier" to see the issue with Sleep(200), it's equally easy in both situation. Does it on your?

Right after UpdatePlatformWindows(), RenderPlatformWindowsXXX is called with the right render.
I think the issue has to do with the fact that there's a delay in swapped contents appearing, and the timing of that swap compared to timing of window size update may be missadjusted. Your suggested workaround is probably a good idea in that context, but I don't think the issue is caused by the thing you think is causing it.

[ocornut]

Additional comments: I think this issue seems new to me, I don't recall seeing when initially implementing multi-viewport for the DX12 backend, and I believe it may also be due to some DX12/Windows version/SDK version/drivers subtleties which I don't quite understand now. Note that DX12 backend also have a "flicker" when dragging a viewport outside, it doesn't appear as far as with other backends, and that's perhaps related to the same underlying thing.

[devkaiwang]

Hi @ocornut ,

Thank you so much for the investigation and explanation.

I have uploaded the testing code for you to check: (Edit: commit updated) devkaiwang@21c4f82

Here is the screenshot of the logs happened when I dragged the border of the "Hello world!" platform window when it was a standalone window out of the main viewport:

As you could see from the log, viewport->Size update did not happen according to the logging I already added to the following code:

if (viewport->PlatformRequestResize)
{
    viewport->Size = viewport->LastPlatformSize = g.PlatformIO.Platform_GetWindowSize(viewport);

    // Debug only:
    char log[1024];
    std::snprintf(log, sizeof(log), "Viewport size update, size=(%u, %u)\n",
        static_cast<uint32_t>(viewport->Size.x), static_cast<uint32_t>(viewport->Size.y));
    OutputDebugStringA(log);
}

If my debug is correct, PlatformRequestResize flag was already cleared by ClearRequestFlags. But by design, as you explained in the Resizing from Platform/OS decorations part, viewport->Size should be updated in the way bullet point ii explained.

To me, it seems ImGui was rendering in old size but the swapchain was in new size, causing the ghosting stretch effect.

I believe your log is partially incorrect (some incorrect recording or manipulation or missing details) ?

I commented out PlatformRequestResize flag clearing code in ClearRequestFlags and had the logs as you saw last time and the update was still one frame behind.

If I add a Sleep(80) or larger value e.g. Sleep(200) you can notice that the error only happens for very short while (at least on my setup), not for 200 ms. So it doesn't make it any "easier" to see the issue with Sleep(200), it's equally easy in both situation. Does it on your?

From my observation, Sleep(200) seems to help reproduce the issue better if I revert the swapchain scaling mode to stretch.

[ocornut]

when I dragged the border of the "Hello world!" platform window when it was a standalone window out of the main viewport:

The resizing borders of PLATFORM WINDOWS don't appears by default, don't appear in your first-post video, and don't appear in devkaiwang@21c4f82#diff-66e3146b58f2d5a61041ee73d7320664a6028a7eccc468df72166822b09ea976 (unless you modified things to set io.ConfigViewportsNoDecoration = false but then you need more logistic to do a "live" resizing) so I am not sure we are talking about the same thing.

You are exactly the situation described in my "Resize from ImGui decorations -> point 3" and your log matches that.
The update of viewport->Size appears in "point 2" of that same description.

---

PS: You can use IMGUI_DEBUG_LOG() to simplify your debugging:

   char log[1024];
    std::snprintf(log, sizeof(log), "Swapchain update, size=(%u, %u)\n",
        static_cast<uint32_t>(size.x), static_cast<uint32_t>(size.y));
    OutputDebugStringA(log);

Becomes:

  IMGUI_DEBUG_LOG("Swapchain update, size=(%u, %u)\n", (uint32_t)size.x, (uint32_t)size.y);

Appears in console and Demo->Tools->Debug Log.

From my observation, Sleep(200) seems to help reproduce the issue better if I revert the swapchain scaling mode to stretch.

It's difficult to compare without a video as it may depends on OS/drivers/GPU and other settings.

[devkaiwang]

Hi @ocornut

Thanks for the reply!

The resizing borders of PLATFORM WINDOWS don't appears by default, don't appear in your first-post video, and don't appear in devkaiwang@21c4f82#diff-66e3146b58f2d5a61041ee73d7320664a6028a7eccc468df72166822b09ea976 (unless you modified things to set io.ConfigViewportsNoDecoration = false but then you need more logistic to do a "live" resizing) so I am not sure we are talking about the same thing.

I'm not sure if I totally understand this but I can use my mouse to resize the platform window as you could see from the video, though the window is not in thick border style of Windows platform. ConfigViewportsNoDecoration is remained as default.
I did the resizing test after the platform window was placed out of the main viewport.

The update of viewport->Size appears in "point 2" of that same description.

This is the problem I'm trying to understand. As you can see from my log, there is no Viewport size update ... log record meaning "point 2" did not happen.

PS: You can use IMGUI_DEBUG_LOG() to simplify your debugging:

Sorry missed this useful function. Thanks for the education.

[ocornut]

resize the platform window as you could see from the video

You are resizing from imgui displayed borders and using imgui resizing logic, not from Windows displayed borders using Windows code. They are two different paths. Windows has its own resizing logic. In your situation it is not used because the Windows borders are hidden

there is no Viewport size update ... log record meaning "point 2" did not happen.

point 2 refer to a different location in the code inside Begin(), where you didn’t add a log entry. If you search for “viewport->Size = “ you will find it.

[ocornut]

I think the issue has to do with the fact that there's a delay in swapped contents appearing, and the timing of that swap compared to timing of window size update may be missadjusted. Your suggested workaround is probably a good idea in that context, but I don't think the issue is caused by the thing you think is causing it.

Interestingly our example app had an extraneous back buffer, which was reduced in 6684984, and reducing it fixed the flickering I described here (which is not your issue, but, read on).

The numFramesInFlight value has an impact on latency. As we aim to support apps using 3 and not 2, the transition of contents between one viewport and the other should be reworked to account for this latency.
Using 3 what happened when destroying a viewport is that the previous viewport gets destroyed "too early" while the main one didn't have time to display the frame with the imgui window moved into it, causing the flickering.

While this is not the same as your issue, my understanding based on previous messages and your description is that your issue had to do with frame backbuffers being queued and appearing too late.

What it effectively means is that user of setups with unusual amount of frame latency should probably delay update of "size" perceived by imgui code, according to this latency.

In single-viewport mode (explaining this for simplicity) we do:

void ImGui_ImplWin32_NewFrame()
{
  RECT rect = { 0, 0, 0, 0 };
  ::GetClientRect(bd->hWnd, &rect);
  io.DisplaySize = ImVec2((float)(rect.right - rect.left), (float)(rect.bottom - rect.top));
  ...

It would need to be changed to a system where the value is "queued" and delayed slightly.

In multi-viewport mode the code:

                if (viewport->PlatformRequestResize)
                    viewport->Size = viewport->LastPlatformSize = g.PlatformIO.Platform_GetWindowSize(viewport);

Should do the same.

Otherwise, it means your code running during Frame N sees Frame N size, and may render e.g. scaled contents, but this scaled contents will only appear on screen on Frame N+1 or N+2, and what appears on Frame N is the contents from Frame N-2 or N-1.

This is purely theoretical, I would likely need to testbed to investigate this more thoroughly.
Of course the "better" solution, if my hypothesis is correct, is that your app should aim to reduce latency.