Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebGPURenderer: Transmission - Backdrop 3/3 #27880

Merged
merged 50 commits into from
Apr 12, 2024
Merged

Conversation

sunag
Copy link
Collaborator

@sunag sunag commented Mar 7, 2024

Related issue: #25903, #26196, #27850

Introduction

material.backdropNode allowed the manipulation of rendered objects as layers, similar to what we can do in Photoshop, for example, to add a node to overlay or any other filter with objects from the back, simply add material.backdropNode = viewportSharedTexture().overlay( someNode ) .

Here some exampels - https://threejs.org/examples/?q=backdrop

This PR introduces the last part of the backdrop implementation. This required some significant changes to the architecture, more details in the schematic below.

Schema

WebGPURenderer - New

image

WebGLRenderer - Current

image

Now all rendering is done on linear-colorspace, and only if color space conversion or tone mapping is needed, a simple post-processing layer is applied to the final rendering. This approach honors the settings defined in rendering regardless of whether the material is set to material.toneMapped=false or material.colorSpaced=false.

Refraction over Refraction

One of the big benefits is that we can have refraction over refraction, as we don't need RenderPass to backdrop work.

WebGPURenderer -> webgpu_loader_gltf_transmission_webgpu

WebGLRenderer -> webgl_loader_gltf_transmission_webgl

webgpu_loader_transmission

WebGPURenderer WebGLRenderer
image image
image image
  • Pros
    • Allows much more flexible management of refraction.
    • No need to render opaque objects twice, which can improve performance in large scenes.
    • -- Allows refraction over refraction. -- >> for other PR
    • Simplifies Shader per object, improves compilation and helps with caching, reducing memory usage.
  • Cons
    • material.toneMapped=false and material.colorSpaced=false are no longer supported, these settings must be defined in the renderer.
    • An additional drawcall, although this is of little significance as it is a QuadMesh.

Repository owner deleted a comment from github-actions bot Mar 7, 2024
@mrdoob
Copy link
Owner

mrdoob commented Mar 8, 2024

Looking good!

Do you mind adding stats.js?
The WebGPU version seems to be much slower than the WebGL version 🤔

@RenaudRohlinger
Copy link
Collaborator

RenaudRohlinger commented Mar 8, 2024

@mrdoob on that not I'm working to add GPU performance monitor as an additional panel via renderer.trackTimestamp. It would be definitely useful for this example. Will be a major improvement for performance analytics in regard to WebGPU vs WebGL!

Related: #27870 #27597

@mrdoob
Copy link
Owner

mrdoob commented Mar 14, 2024

Another thing I noticed is that the transmission seems to be one frame behind? (top-left sphere show the issue clearly)

Screen.Recording.2024-03-14.at.2.37.11.PM.mov

@mrdoob
Copy link
Owner

mrdoob commented Apr 5, 2024

Let me know when this is ready to be reviewed again. The links in the first post no longer work.

@sunag
Copy link
Collaborator Author

sunag commented Apr 5, 2024

Let me know when this is ready to be reviewed again. The links in the first post no longer work.

Yes! I think we can review it again.

@sunag
Copy link
Collaborator Author

sunag commented Apr 5, 2024

New links... I removed the old examples but I can recovery if necessary

these are the new
https://raw.githack.com/sunag/three.js/dev-postprocessing/examples/index.html?q=webgpu%20transmission#webgpu_loader_gltf_transmission

image

@Mugen87
Copy link
Collaborator

Mugen87 commented Apr 5, 2024

I've noticed a minor thing in webgpu_materials_transmission. Compared to webgl_materials_physical_transmission, it seems the reflections of the inner faces are not visible through the transmissive front side. It's a bit hard to explain but if you open both examples and compare the refraction on the sphere, some details in webgpu_materials_transmission are missing.

@mrdoob
Copy link
Owner

mrdoob commented Apr 5, 2024

@sunag

New links... I removed the old examples but I can recovery if necessary

Yes please 🙏

@sunag
Copy link
Collaborator Author

sunag commented Apr 5, 2024

@Mugen87 I think that this is the same problem mentioned here #27880 (comment) referring to DoubleSided,

I think it's better to do this in another PR, this one is already very large, and makes very significant changes to the engine.

Explaining better this #27880 (comment) comment, to continue the agonotic renderer, the Renderer needs to have all the necessary resources so that we do not execute this process in core, for example in WebGLRenderer there is a very specific double pass when transmission have DoubleSided materials activated, where all transmission materials are rendered on the back-sided already with mipmaps applied of opaque materials, and after back-sided be rendered the mipmaps is generated again and render all transmission materials on the front-sided allowing you to see one face inside the other, this is possible following this approach, but I believe that due to the scope of the update it is better to leave it in another PR , and this will only really be necessary if a new refraction over refraction approach does not meet the performance needs.

@sunag
Copy link
Collaborator Author

sunag commented Apr 5, 2024

@mrdoob The same code was used, just this change based from url parameter: renderer = webgpu ? new WebGPURenderer() : new WebGLRenderer()

WebGPURenderer with WebGLBackend fallback
https://raw.githack.com/sunag/three.js/dev-postprocessing-examples/examples/transmission.html?renderer=webgpu

WebGLRenderer
https://raw.githack.com/sunag/three.js/dev-postprocessing-examples/examples/transmission.html?renderer=webgl

@sunag
Copy link
Collaborator Author

sunag commented Apr 6, 2024

@mrdoob I think performance will no longer be a problem, I just did some tests on the M1 in Chrome with the frame-rate-limit disabled and got an average of 172fps with WebGPURenderer vs 83fps using WebGLRenderer, this difference should increase if the scene had more opaque materials.

Only the points mentioned above that I must resolve in other PRs.

@mrdoob
Copy link
Owner

mrdoob commented Apr 11, 2024

Looks good!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants