Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebGPURenderer: Add shadowNode to NodeMaterial #27625

Merged
merged 9 commits into from
Jan 27, 2024

Conversation

RenaudRohlinger
Copy link
Collaborator

This PR adds a way to control the shadow at a fragment level (discard pixels for example).

Updated webgpu_shadowmap example:

Screenshot 2024-01-26 at 18 19 44

This contribution is funded by Utsubo

@gkjohnson
Copy link
Collaborator

This is an area where I thought nodes could provide simpler and more intuitive behavior. One common issue for users at the moment is that custom shaders that discard or perturb vertices don't just render a correct shadow map. I'm wondering if the rendered shadow object could just implicitly use the discard logic used in the color node path rather than requiring the use to remake it for the shadow node.

And if a user wants a different custom shadow behavior then they can plug it in.

Comment on lines 115 to 121
materialCustomShadow.shadowNode = tslFn( () => {

discardNode.discard();

return vec4( 1. );

} )();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that when commenting only this shadowNode line out neither the object surface nor the shadow have discard behavior even those the "colorNode" is still calling the discard node. Is this correct?

Copy link
Collaborator Author

@RenaudRohlinger RenaudRohlinger Jan 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, it's because it wasn't a NodeMaterial (it should be MeshPhongNodeMaterial instead of MeshPhongMaterial)

@RenaudRohlinger
Copy link
Collaborator Author

RenaudRohlinger commented Jan 26, 2024

One common issue for users at the moment is that custom shaders that discard or perturb vertices don't just render a correct shadow map

Currently, the shadow map system already inherits from the positionNode so it does get aligned with the custom transformation of the vertices. I tried to follow that logic in the renderObject method of the Renderer but there is no way to decide which part of the fragment should be used for that matter (color, output, opacity...).

So vertex wise shadow maps are already good and aligned with the custom node material (with positionNode being shared), and fragment wise we can't just decide on a specific node to duplicate its discarding behavior nor use the full composition of all the node fragments as it would be too heavy for a shadow. I think the best current solution would be to simply add a shadowNode to handle the fragment part. /cc @gkjohnson

What are your thoughts on this @sunag?

@LeviPesin
Copy link
Contributor

fragment wise we can't just decide on a specific node to duplicate its discarding behavior nor use the full composition of all the node fragments as it would be too heavy for a shadow

Can you please explain this more? I don't really understand...

@sunag
Copy link
Collaborator

sunag commented Jan 26, 2024

I think that material.shadowNode in material should be related with the shadowNode of AnalyticLightNode, in this case I believe that materialCustomShadow.shadowNode = vec4( color( 0x0000ff ), 1 ) should return a blue shadow, this could be useful in creation of customized translucent materials.

@sunag
Copy link
Collaborator

sunag commented Jan 26, 2024

I added opacity and shadow color by material in this PR, I think it was closer to what I mentioned here #27625 (comment).

image

@sunag
Copy link
Collaborator

sunag commented Jan 26, 2024

I'm going to make some revisions, I think we can launch it in this release :)

@sunag sunag added this to the r161 milestone Jan 26, 2024
@sunag sunag merged commit 964d35d into mrdoob:dev Jan 27, 2024
11 checks passed
@gkjohnson
Copy link
Collaborator

@RenaudRohlinger

there is no way to decide which part of the fragment should be used for that matter (color, output, opacity...).

Admittedly I haven't used the nodes system extensively but one of the benefits of nodes as I've seen is the simpler ability to traverse the graph of shader operations. If "discard" operations are modeled as terminating nodes here (ie a node with no descendants), then it should be possible to find all those "discard" nodes, prune any nodes & connections that don't feed into them upstream, and generate a node graph of operations relevant for the shadow map. Am I misunderstanding something?

It's definitely something for the future and the approach in this PR is good and worth having for custom shadow behavior. But I think it would be worth tracking the above since I think this is something that a node material system should enable.

@RenaudRohlinger
Copy link
Collaborator Author

Thanks @sunag it's a lot better this way!

I see @gkjohnson, it makes a lot of sense. Definitely something we can explore in the future 👍

@LeviPesin
Copy link
Contributor

LeviPesin commented Jan 28, 2024

If "discard" operations are modeled as terminating nodes here (ie a node with no descendants), then it should be possible to find all those "discard" nodes, prune any nodes & connections that don't feed into them upstream, and generate a node graph of operations relevant for the shadow map.

This doesn't really sound similar to what internally happens in the Nodes system... The closest to this which I can recall is Node.analyze() step, but I don't really imagine how to implement something like this with it.

@gkjohnson
Copy link
Collaborator

I guess what I'm getting at is that notionally this is something that should be possible when you have a shader graph - the ability to procedurally analyze, modify, and build sibling graphs is powerful. This can be used for optimization (auto deduping of nodes), material model replacement (ie change from standard to phong or custom by without losing the inputs), etc.

Maybe @sunag has thoughts on how possible this is now?

@sunag
Copy link
Collaborator

sunag commented Jan 29, 2024

@gkjohnson I agree, I think we have already done some things in this regard, for example:

  • Node can handle a graph shader node instances, this means that if there are two different objects like new NormalNode( NormalNode.VIEW ) in different connections, only the first one will generate the code and the other will share the same code.

  • Node automatically generates variables as it is used as input two or more times to avoid duplicate calculations.

  • We can generate isolated code in different contexts from a single Node. This allows, for example, the same material.envNode to be created individually for radiance and irradiance.

  • Node inputs like material.colorNode could be used in any other material or any other input like material.alphaNode, variable conversions are done automatically. I think that in principle it would be the same thing between Standard and Phong material.

There are probably more things that I didn't remember, but there are certainly things to be developed in this regard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants