Async Shader Compilation (KHR_parallel_shader_compile)

[Maksims]

There is a work going on for KHR_parallel_shader_compile extension:
Specs: https://www.khronos.org/registry/webgl/extensions/KHR_parallel_shader_compile/
Chrome Issue Tracker: https://bugs.chromium.org/p/chromium/issues/detail?id=849576

It allows to send shader to compiler without locking thread and check on compilation status later on.

That will speed up shader compilation on overall as well as open ability to compile shaders without thread lockup as it is now. From benchmarks even sync compilation path have already ~50% compilation time improvement.

Would be great to think about integrating it into the Engine once available.

[willeastcott]

Already done:

https://github.com/playcanvas/engine/blob/master/src/graphics/device.js#L699

[Maksims]

But it is a bit more involved, there is a setting for number of threads can be used. And then it is only beneficial when instead of querying LINK_STATUS of a Program, it should query ext.COMPLETION_STATUS_KHR until it returns true, only then should query LINK_STATUS.

This can't be just plugged in, and will require engine to take it in account. And some behaviour development.
For example:

1. Start precompiling shaders in the beginning of preloading phase.
2. Allowing developer to specify material to be async or sync which will lead to changes that regenerate ubershader to be immediate with thread stall or asynchronous.
3. What is API to compile shader, will it provide a way to start compilation and then ignore it if shader changed on the way, so all those API bits regarding shader code.

[willeastcott]

Ah yeah, completion status needs updating.

However, the implementation should select a sensible number of threads, as hinted by the comment in the spec sample code:

    var ext = gl.getExtension('KHR_parallel_shader_compile');
    if (ext) {
      // Just for demo of API usage. Generally it's not needed unless you really
      // want to override the implementation-specific maximum.
      var threads = gl.getParameter(ext.MAX_SHADER_COMPILER_THREADS_KHR);
      ext.maxShaderCompilerThreadsKHR(Math.max(2, threads));
    }

[Maksims]

Status of this extension, been merged to latest ANGLE, so on Windows we will have it soon: gpuweb/gpuweb#144 (comment)

[Maksims]

Since this extension been promoted to Community Approved status, it has been implemented in some browsers. So my Chrome 76, can already show it supports that extension.

Implementation

Based on previous discussions, here are some thoughts of implementation:

1. By default all user-created materials will be async.
2. Engine, picker, depth, post-effects, etc, shaders will be non-async.
3. Editor will have checkbox "async shader" on material inspector, so disabling it will force sync path.
4. Add clear documentation and info, that if material has async shader: rendering object might not render it as shader is not compiled yet.
5. Disabling async on material.shader will ensure object will be rendered and will freeze thread if shader is not compiled.

Considerations

Currently I've tested different projects, where if shader is not compiled, I just skip drawcall, this leads to some popping. Additional issues related to this:

1. Rendering behaviour is changed, as when shader is not linked yet, it cannot render object. So either we simply don't render it, or we render it with some standard material, like in case of texture popping. This even more apparent on first frame load, as it will on first frame, more likely show only skybox, and then few frames later show the rest.
2. Editor first/subsequent renders. Editor does not render scene all the time, but triggers render function manually when Editor believes there is a need for render. So in order to make sure renders happen when needed, there should be even handler when shader is compiled. But there is no callback from WebGL. And having a counter on shaders not force linked - wont work, as next render that particular shader might not be required. One option is, to have and Editor list of shaders that are not force linked, but been requested to be used. So then in Editor, we will check periodically if they are now compiled, and if so, then we request rendering frame. This will avoid freezes and will render again once shader is compiled.
3. Baking/Single Renders - when baking stuff, shaders have to be compiled, so rendering with forced linking might be required. But what controls we will provide? Should we provide a flag, when flipped on, it will ensure that next frame all shaders will force link?

Behaviour change

Due to skip of rendering, there is blinking (popping) now of objects when their shaders compiling.
How to avoid it, is not an easy question to answer.
Few options to explore:

1. Keep them not rendered and then let the reappear.
2. Apply standard material on them.
3. Try to apply previous shader while new is compiling. This might be complicated, especially if model with different uniforms or parameters are changed at the same time, they might create conflicts of things.

Tests

This will include time spent on blocked shader compilation. This will be preload time, and then time spent during application working.

1. Seemore 80% less freezing during runtime.
   Preload (old) ~113ms
   Sync (old) ~485ms
   Preload (new) ~101ms
   Async (new) ~95ms

2. Sponza Lightmap 55% less freezing during runtime
   Preload (old) ~651ms
   Sync (old) ~541ms
   Preload (new) ~440ms
   Async (new) ~242ms

3. Gunsmith 62% less freezing during runtime
   Old ~202ms
   New ~77ms

Looks like great results.
Will update once progressed further.

[willeastcott]

1. I would say we don't render it. The shader is potentially doing more than coloring a fragment differently - a vertex shader might be dramatically transforming a vertex away from a standard static shader. I wouldn't want my characters to render in T-pose, for example, while the skinning shader is compiling.
2. Editor is interesting. We could just keep the current behavior in the Editor but the freezing if the UI on scene load is pretty nasty (despite me getting completely used to that over the years). The Editor could do something like:

// Init somewhere
this.compiled = 0;

setTimeout(function () {
    var currCompiled = 0;
    this.app.graphicsDevice.shaders.forEach(function (shader) {
        if (shader.ready) {
            currCompiled++;
        }
    });
    if (currCompiled > this.compiled) {
        editor.render();
        this.compiled = currCompiled;
    }
}.bind(this), 250);

Seems pretty low cost to me.
3. I'm not quite sure what you mean by controls here. Do you mean in the API or in the Editor?

Another point: there's an internal function that is still internal (i.e. not in the public API ref).

https://github.com/playcanvas/engine/blob/master/src/graphics/program-library.js#L95

If you call that from the console, it'll serialize all shader definitions to a JS file. Add that generated script to your project and all your shaders will precompile. Seems that that, combined with this implementation are a perfect match.

[Maksims]

2. The Editor could do something like:

It's a bit more complicated.
There is a case, when compilation is started, then this tick it is async checked if it is compiled, it is not, then next frame mesh with that shader is hidden, so it never calls useShader on it again, so will never find out if it has compiled. This will lead to some shaders to have ! ready flag.

In Editor it is even another story. When frame happens, it checks if shader is compiled, if not, it might compile by the next frame. But in Editor we do not call next frame.

One easy solution, is to have an internal counter, of how many shaders were encountered that are not compiled. Editor simply checks if this counter is > 0, then queues next frame render. Simple, fast, reliable.

3. I'm not quite sure what you mean by controls here. Do you mean in the API or in the Editor?

Here is a scenario: you have a room, and want to render cubemap from middle of it. But you need to show/hide some objects just for that cubemap render. If you do this, with async shaders, then some objects will not render, and user don't have a control over it.
One way, is to expose a flag, if true it will enforce sync shader compilation. That way user can have control over it.

With those 2 things sorted, I believe it will be ready for merge.

[Maksims]

I've added graphicsDevice._shaderAsyncCompilationsFrame property. It will increment when there are async shaders were compiling during last frame.
So Editor can now simply check after frame rendering, if this property is > 0 then schedule render frame again.

[willeastcott]

Nice, sounds sensible.

[Maksims]

Added public property GraphicsDevice.asyncShaderCompilation which is available to developer, so he/she can disable async shader compilation if needed. Like in cases where meshes/materials switching required just before rendering scene into texture, so have to ensure all meshes are rendered. By default it is true.

I believe it is ready to be merged.