Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shader lazy load #382

Merged
merged 47 commits into from
Oct 24, 2023
Merged

Shader lazy load #382

merged 47 commits into from
Oct 24, 2023

Conversation

roeas
Copy link
Contributor

@roeas roeas commented Sep 11, 2023

  • ShaderCollections
  • Refactor RenderContext
  • Refactor Renderer

Compiling and loading shaders like this, instead of compiling all shader variants at once:
Snipaste_2023-09-22_16-57-58

Heres an example of the Renderer frame:

void WorldRenderer::Init()
{
	constexpr StringCrc programCrc = StringCrc("WorldProgram");
	GetRenderContext()->RegisterShaderProgram(programCrc, { "vs_PBR", "fs_PBR" });
        // ...
}

void WorldRenderer::Warmup()
{
	GetRenderContext()->CreateUniform(lutSampler, bgfx::UniformType::Sampler);
	GetRenderContext()->CreateTexture(lutTexture);
        // GetRenderContext()->UploadShaderProgram("WorldProgram");
	// ...
}

void WorldRenderer::Render(float deltaTime)
{
	// ...
	// GetRenderContext()->Submit(GetViewID(), "WorldProgram");
	GetRenderContext()->Submit(GetViewID(), "WorldProgram", pMaterialComponent->GetFeaturesCombine());
}

Click here if you want to check every variant of your uber shader compiling.
image

Copy link
Contributor

@T-rvw T-rvw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, it looks that you limit Init to create CPU data and add CreateGraphicsResources interface to submit cpu data to gpu.

  1. [Optional] Submit maybe a better name to present CreateGraphicsResources?
  2. [Optional] You can wrap all renderer or renderpass logics to a new data structure such as engine::RenderPipeline and EditorApp can simply hold m_editorRenderPipeline and m_engineRenderPipeline. The pipeline concept is a series of Renderer/RenderPass in order. Multiple RenderPipelines(Rasterize, RayTrace, Compute, ...) contribute to the final rendering world.
  3. Currently, we have an user-visible entity named shader collection. It seems not very reasonable.

@roeas
Copy link
Contributor Author

roeas commented Sep 13, 2023

Overall, it looks that you limit Init to create CPU data and add CreateGraphicsResources interface to submit cpu data to gpu.

  1. [Optional] Submit maybe a better name to present CreateGraphicsResources?
  2. [Optional] You can wrap all renderer or renderpass logics to a new data structure such as engine::RenderPipeline and EditorApp can simply hold m_editorRenderPipeline and m_engineRenderPipeline. The pipeline concept is a series of Renderer/RenderPass in order. Multiple RenderPipelines(Rasterize, RayTrace, Compute, ...) contribute to the final rendering world.
  3. Currently, we have an user-visible entity named shader collection. It seems not very reasonable.
  1. Sounds reasonable.
  2. OK, I'll apply this concept after these shader loading and compiling stuff done.
  3. I use this entity to implement functionality similar to this document https://docs.unity3d.com/2023.2/Documentation/Manual/shader-variant-collections.html.
    The SVC needs to be stored in the project file right?
    Still under consideration by me.

@roeas
Copy link
Contributor Author

roeas commented Sep 13, 2023

Indeed it is a bit strange to think of SVC as a component.

@T-rvw
Copy link
Contributor

T-rvw commented Sep 13, 2023

In project serialization, the main part is Entity/Component data but also allow to add other data such as SVC. It seems an asset, not a component in the unity document.

@T-rvw T-rvw added the WIP Work In Progress label Sep 15, 2023
@T-rvw T-rvw force-pushed the main branch 2 times, most recently from feb0da8 to fd37097 Compare September 18, 2023 17:56
Comment on lines -12 to +24
Projects/Shared
Projects/Shared/*
Projects/Test/*
!Projects/Test/*/
Projects/Test/Textures/*
!Projects/Test/Textures/*/

# Precompiled Shader
!Projects/Shared/BuiltInShaders
Projects/Shared/BuiltInShaders/*
!Projects/Shared/BuiltInShaders/Direct3D11
Projects/Shared/BuiltInShaders/Direct3D11/*
!Projects/Shared/BuiltInShaders/Direct3D11/vs_imgui.bin
!Projects/Shared/BuiltInShaders/Direct3D11/fs_imgui.bin
Copy link
Contributor Author

@roeas roeas Sep 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upload vs_imgui.bin and fs_imgui.bin to github.
Seems that i cant just write something like

Projects/Shared/*
!Projects/Shared/BuiltInShaders/Direct3D11/vs_imgui.bin

to upload vs_imgui.bin but ignore other files under Shared.

@roeas roeas changed the title Shader varient collections. Shader lazy load and refactor RenderContext. Sep 22, 2023
@roeas roeas changed the title Shader lazy load and refactor RenderContext. Shader lazy load Sep 22, 2023
@T-rvw
Copy link
Contributor

T-rvw commented Sep 27, 2023

You can add a launch option which let editor pre-compile all shaders? Sometimes it will be helpful. For example, a user doesn't edit any shader, just make a game level which doesn't need to wait in a random timepoint.

@T-rvw T-rvw merged commit efd2d33 into main Oct 24, 2023
3 checks passed
@T-rvw T-rvw deleted the collection branch October 24, 2023 12:44
@T-rvw T-rvw removed the WIP Work In Progress label Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants