ImGui::Image renders a font atlas instead of a multi-sampled texture

[sebastianzander]

Version/Branch of Dear ImGui:

Version: v1.80 WIP
Branch: docking

Back-end/Renderer/Compiler/OS

Back-ends: imgui_impl_glfw.cpp + imgui_impl_opengl3.cpp (OpenGL 4.6 on a NVIDIA GeForce RTX 2060)
Compiler: Microsoft Visual Studio Enterprise 2017 (15.9)
Operating System: Windows 10

My Issue:

I render to a framebuffer in order to have multiple scene view windows of my 3D scene and in order to render the color or the depth buffer for reasons of debugging and research. It worked well without multi-sampled textures but I want to get rid of aliasing. That's why I switched my normal GL_TEXTURE_2D implementation to GL_TEXTURE_2D_MULTISAMPLE. I check the framebuffer status, OpenGL says everything is fine with it.

When I use ImGui::Image as before (no changes here) I see a font atlas (presumably from ImGui) instead of my framebuffer (color or depth attachment).

Screenshots

Standalone, minimal, complete and verifiable example:

// what I do to create or invalidate my framebuffer
int maxSamples;
glGetIntegerv(GL_MAX_SAMPLES, &maxSamples);
int samples = std::min(maxSamples, 16);

if(!m_framebufferId) {
    glCreateFramebuffers(1, &m_framebufferId);
    glCreateTextures(GL_TEXTURE_2D_MULTISAMPLE, 1, &m_colorAttachment);
    glCreateTextures(GL_TEXTURE_2D_MULTISAMPLE, 1, &m_depthAttachment);
}

glBindFramebuffer(GL_FRAMEBUFFER, m_framebufferId);

glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, m_colorAttachment);
glTexImage2DMultisample(GL_TEXTURE_2D_MULTISAMPLE, samples, GL_RGBA8, m_viewportWidth, m_viewportHeight, GL_TRUE);
glTexParameteri(GL_TEXTURE_2D_MULTISAMPLE, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D_MULTISAMPLE, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D_MULTISAMPLE, m_colorAttachment, 0);

glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, m_depthAttachment);
glTexStorage2DMultisample(GL_TEXTURE_2D_MULTISAMPLE, samples, GL_DEPTH32F_STENCIL8, m_viewportWidth, m_viewportHeight, GL_TRUE);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_TEXTURE_2D_MULTISAMPLE, m_depthAttachment, 0);

CheckFramebufferStatus();

glBindFramebuffer(GL_FRAMEBUFFER, 0);

// what I do to draw the image
ImGui::Image((ImTextureID)(intptr_t)drawImage, viewportSize, ImVec2 { 0.f, 1.f }, ImVec2 { 1.f, 0.f });
// where drawImage is either m_colorAttachment or m_depthAttachment

As stated earlier, I only replaced GL_TEXTURE_2D with GL_TEXTURE_2D_MULTISAMPLE, glTexImage2D with glTexImage2DMultisample and glTexStorage2D with glTexStorage2DMultisample (the arguments changed of course)

[ocornut]

Hello,

You can see how the render loop is setup in imgui_impl_opengl3.cpp
I don't know the OpenGL specs but I am assuming that it calling glBindTexture(GL_TEXTURE_2D, pcmd->TextureId); won't provide the intended binding for reading back from that texture.

Normally we use draw callback for manipulating the render state from outside but here specifically since that glBindTexture() is in the main path it is a little tricky.

• My suggestion would be to copy the ImGui_ImplOpenGL3_RenderDrawData() function in your codebase and have a modified one that decide how to bind based of some other data. You can redefine ImTextureID as a higher-level type, or have your render loop find out the texture type to do the binding.

• I think the approach used in Dx12/vulkan texture management & custom binding #2697 of enabling users to override texture binding while reusing most of the existing draw loop is something that would solve this problem and I am committed to making it happen eventually.

• Since you are drawing a simple quad you could also use ImDrawList::AddCallback() instead of Image() and have your callback perform the binding + the rendering of a simple quad.

[sebastianzander]

Hey, thank you for your response. I tried your suggestion, but ImGui_ImplOpenGL3_RenderDrawData() depends on static void ImGui_ImplOpenGL3_SetupRenderState() and after I copied that method into my codebase I now miss more and more variables or method from its original module.

I would like to try the third option, use ImDrawList::AddCallback() and try to emulate what Dear ImGui does in ImGui::Image() and subsequently in ImDrawList::AddImage() but I run into similar problems regarding accessibility or visibility of variables and methods. Or are you suggesting that I would have to use native OpenGL methods in my callback and use my own shader?

[ocornut]

Or are you suggesting that I would have to use native OpenGL methods in my callback and use my own shader?

You can use native OpenGL methods for submitting a simple vertex buffer, but you can use the ImDrawVert type and use the existing ImGui shader if it works for you

[sebastianzander]

I have tried so much by now regarding ImDrawList::AddCallback() but didn't achieve anything. What I must and mustn't call and do in my callback remains a mystery to me. Whenever I try to move the (dockable) ImGui window or try to open any ImGui menu my program crashes with an Access violation reading location 0x00000000 thrown in nvoglv32.dll. The window moves for a couple of pixels before the program crashes.
Since I have no PDB for the OpenGL library I can only trace the exception back to the glDrawElementsBaseVertex() call within these lines in imgui_impl_opengl3.cpp:

#ifdef IMGUI_IMPL_OPENGL_MAY_HAVE_VTX_OFFSET
    if (g_GlVersion >= 320)
        glDrawElementsBaseVertex(GL_TRIANGLES, (GLsizei)pcmd->ElemCount, sizeof(ImDrawIdx) == 2 ? GL_UNSIGNED_SHORT : GL_UNSIGNED_INT, (void*)(intptr_t)(pcmd->IdxOffset * sizeof(ImDrawIdx)), (GLint)pcmd->VtxOffset);
    else
#endif

But this is aside of the if (pcmd->UserCallback != NULL)-branch (in fact, it is the else-branch), so I assume my callback somehow compromises the internal ImGui state? But how is that possible and what would I have to do to fix this?

This is what my callback currently looks like (it's a static member function and works just as good as a non-member friend function):

void Viewport::DrawViewportQuad(const ImDrawList* parentList, const ImDrawCmd* cmd)
{
    ViewportDrawData* data = (ViewportDrawData*)cmd->UserCallbackData;
    Viewport* viewport = data->viewport;

    static GLint u_proj = viewport->m_viewportQuadShader->FindUniform("u_proj");
    static GLint u_drawMode = viewport->m_viewportQuadShader->FindUniform("u_drawMode");
    static GLint u_colorBuffer = viewport->m_viewportQuadShader->FindUniform("u_colorBuffer");
    static GLint u_depthBuffer = viewport->m_viewportQuadShader->FindUniform("u_depthBuffer");

    // remember important last states
    GLint lastProgram = 0; glGetIntegerv(GL_CURRENT_PROGRAM, &lastProgram);
    GLboolean lastSampleShading = 0; glGetBooleanv(GL_SAMPLE_SHADING, &lastSampleShading);

    viewport->m_viewportQuadShader->BindShader();

    ImDrawData* drawData = ImGui::GetDrawData();
    float L = drawData->DisplayPos.x;
    float R = drawData->DisplayPos.x + drawData->DisplaySize.x;
    float T = drawData->DisplayPos.y;
    float B = drawData->DisplayPos.y + drawData->DisplaySize.y;
    
    const float viewportProjection[4][4] = {
        { 2.0f/(R-L),   0.0f,         0.0f,   0.0f },
        { 0.0f,         2.0f/(T-B),   0.0f,   0.0f },
        { 0.0f,         0.0f,        -1.0f,   0.0f },
        { (R+L)/(L-R),  (T+B)/(B-T),  0.0f,   1.0f },
    };

    glUniformMatrix4fv(u_proj, 1, GL_FALSE, &viewportProjection[0][0]);
    
    // bind framebuffer textures
    glActiveTexture(GL_TEXTURE0);
    glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, viewport->m_multisampledColorAttachment);
    glActiveTexture(GL_TEXTURE1);
    glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, viewport->m_multisampledDepthAttachment);

    glUniform1i(u_drawMode, viewport->m_drawMode);
    glUniform1i(u_colorBuffer, 0);
    glUniform1i(u_depthBuffer, 1);

    // set up required state
    glEnable(GL_SAMPLE_SHADING);

    // draw viewport quad
    glBindVertexArray(data->vertexArray);
    glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_INT, 0);
    glBindVertexArray(0);

    glActiveTexture(GL_TEXTURE0);
    glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, 0);
    glActiveTexture(GL_TEXTURE1);
    glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, 0);
    glActiveTexture(GL_TEXTURE0);
    viewport->m_viewportQuadShader->UnbindShader();

    // restore last state
    if(!lastSampleShading)
        glDisable(GL_SAMPLE_SHADING);

    glUseProgram(lastProgram);
}

What I do regarding updating the vertex buffer whenever I resize my ImGui viewport window:

void Viewport::ResizeViewportQuad()
{
    ImVec4& rect = m_viewportDrawData.rect;

    glGenVertexArrays(1, &m_viewportDrawData.vertexArray);
    glBindVertexArray(m_viewportDrawData.vertexArray);

    // defines the vertices of the viewport quad
    const float vertexBufferData[] = {
        // positions                            // texcoords
        rect.x + rect.z, rect.y + rect.w, 0.f,  1.f, 1.f,
        rect.x + rect.z, rect.y, 0.f,           1.f, 0.f,
        rect.x, rect.y, 0.f,                    0.f, 0.f,
        rect.x, rect.y + rect.w, 0.f,           0.f, 1.f,
    };

    static const uint32_t indexBufferData[] = {
        0, 1, 3,
        1, 2, 3
    };

    glGenBuffers(1, &m_viewportDrawData.vertexBuffer);
    glGenBuffers(1, &m_viewportDrawData.indexBuffer);

    glBindBuffer(GL_ARRAY_BUFFER, m_viewportDrawData.vertexBuffer);
    glBufferData(GL_ARRAY_BUFFER, sizeof(vertexBufferData), vertexBufferData, GL_STATIC_DRAW);

    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, m_viewportDrawData.indexBuffer);
    glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(indexBufferData), indexBufferData, GL_STATIC_DRAW);

    // position attribute
    glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 5 * sizeof(float), (void*)0);
    glEnableVertexAttribArray(0);

    // texture coord attribute
    glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 5 * sizeof(float), (void*)(3 * sizeof(float)));
    glEnableVertexAttribArray(1);
}

And here is how I add the callback (with complete context):

void Viewport::Draw()
{
    const ImVec2 labelPadding { 0.f, 3.f };
    static float imguiCursorY = 0.f;

    ImGui::PushID(ImGui::GetID(this));
    ImGui::PushStyleVar(ImGuiStyleVar_WindowPadding, ImVec2(0.f, 0.f));

#define IMGUI_MENUITEM_LEFT(label, func, ...) (ImGui::SetCursorPosY((imguiCursorY = ImGui::GetCursorPosY()) + \
    labelPadding.y), ImGui::Text(label), ImGui::SameLine(), ImGui::SetCursorPosY(imguiCursorY), \
    ImGui::func("## " label, __VA_ARGS__))
    
    ImGui::SetNextWindowSize(ImVec2((float)m_viewportWidth, (float)m_viewportHeight), ImGuiCond_FirstUseEver);
    if(ImGui::Begin("Scene", nullptr, ImGuiWindowFlags_MenuBar))
    {
        ImGui::PopStyleVar();
        ImGui::PushStyleColor(ImGuiCol_MenuBarBg, ImVec4(0.f, 0.f, 0.f, 0.f));
        if(ImGui::BeginMenuBar())
        {
            if(ImGui::BeginMenu("View"))
            {
                bool isColorDrawMode = m_drawMode == DM_Color;
                if(ImGui::MenuItem("Color Buffer", "Ctrl+1", &isColorDrawMode))
                    m_drawMode = DM_Color;

                bool isDepthDrawMode = m_drawMode == DM_Depth;
                if(ImGui::MenuItem("Depth Buffer", "Ctrl+2", &isDepthDrawMode))
                    m_drawMode = DM_Depth;

                /*bool isStencilDrawMode = m_drawMode == DM_Stencil;
                if(ImGui::MenuItem("Stencil Buffer", nullptr, &isStencilDrawMode))
                    m_drawMode = DM_Stencil;*/

                ImGui::Separator();

                if(ms_maxSampleCount > 1 && ImGui::BeginMenu("Multisampling"))
                {
                    if(ImGui::MenuItem("Multisampling Enabled", "Ctrl+Shift+M", &m_bMultisamplingEnabled))
                        Invalidate();

                    int sampleCount = m_sampleCount;
                    if(ms_maxSampleCount > 2 && IMGUI_MENUITEM_LEFT("Sample Count", InputInt, (int*)&sampleCount, 2, 2)) {
                        SetDesiredSampleCount(sampleCount);
                    }

                    ImGui::EndMenu();
                }

                ImGui::EndMenu();
            }

            ImGui::EndMenuBar();
        }
        ImGui::PopStyleColor();

        ImVec2 screenPos = ImGui::GetCursorScreenPos();
        ImVec2 viewportSize = ImGui::GetContentRegionAvail();
        if((uint32_t)viewportSize.x != m_viewportWidth || (uint32_t)viewportSize.y != m_viewportHeight)
            Resize((uint32_t)viewportSize.x, (uint32_t)viewportSize.y);

        m_viewportDrawData.rect = { screenPos.x, screenPos.y, viewportSize.x, viewportSize.y };
        ImGui::GetWindowDrawList()->AddCallback(DrawViewportQuad, &m_viewportDrawData);
        
        ImGui::End();
    }
    else
        ImGui::PopStyleVar();

    ImGui::PopID();
}

What happens in Resize() isn't essential here - it basically resizes my frame, color and depth buffers according to the new viewport size and makes a call to ResizeViewportQuad() at last.

[sebastianzander]

I found out that it is my responsibility to restore the state of the last vertex array/buffer bindings at the end of my callback. That resolved the Access violation reading location 0x00000000 mentioned above.

Now however, what my callback renders to the screen can only be seen when I move the ImGui viewport window at certain positions such that the upper left corner of it is outside of my OpenGL main window:

For an explanation: at first you see my viewport use ImGui::Image() with framebuffer blitting in the background, then I switch to "custom callback" in the viewport menu. This is when the viewport turns grey and the scene is only rendered when the viewport window is moved to certain positions.

I can't simply use ImGui::Image() with framebuffer blitting since I have to modify and scale depth buffer values for better visualization. A shader will also be necessary when I want to apply post-processing effects in the future.

Another off topic question: Is it possible to make the menu background in the viewport transparent? I tried pushing different style colors with different alpha values but no luck so far. Ideally I would like to try make it fully transparent and be on top of my viewport image.

[sebastianzander]

Any update regarding a possible solution or at least a confirmation that it is a bug?

[ocornut]

Well its a bug but since all the rendering is in your hands at this point it is a bug in your code.

Have you tried using a graphics debugger such as Renderdoc to debug your issue and find what’s causing the image to disappear?

Once you solve it, if you can suggest a solution we could implement on the backend side I am happy to hear it.

[CheerWizard]

Hi!
I have completely identical issue btw. Maybe someone has already an answer to this? As soon as I will find proper solution for this, I will notify you here. So, please don't close this issue yet! ;)

[CheerWizard]

@sebastianzander I have simply used blit buffers for resolving GL_TEXTURE_2D_MULTISAMPLE into GL_TEXTURE_2D state. However, as I understand, it won't fit all your needs. Other solution I still don't have, it needs more investigation.

[Rockstar50373]

#3591 (comment)

[ocornut]

Pushed some changes e94f95d (+ f890d85) to formalize giving access to more data from callbacks (see #6969 (comment)) so I wonder if some of this may be easier to solve if we expose data from the OpenGL render function?

I was also working on a more general/wider version of #2697 to allow differentiating "internal textures" (created/managed by backend) from user texture. But a simpler version in the form of a single callback like in #2697 may facilitate this.

@sebastianzander I don't presume you still have this code in place and running? (it's been a while)
If you do I would be happy to have another look together and seeing how we can simplify the solution.

[ocornut]

I found out that it is my responsibility to restore the state of the last vertex array/buffer bindings at the end of my callback.

You may in theory also submit another callback using ImDrawCallback_ResetRenderState.

Basically if you still have this code running and around, I would be happy to see the code and further discuss simpler solutions.

[sebastianzander]

@sebastianzander I don't presume you still have this code in place and running? (it's been a while) If you do I would be happy to have another look together and seeing how we can simplify the solution.

Hi @ocornut, no sorry I don't have it running anymore. It was initially a hobbyist project and turned out to even help me with my thesis back in 2020, but I haven't done any OpenGL stuff since. In the meantime I got a new computer, so I don't have any OpenGL stuff installed, and I don't think that I will find spare time to get it running again.