-
Notifications
You must be signed in to change notification settings - Fork 351
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft : Implementing structures. #1684
Draft : Implementing structures. #1684
Conversation
The code here is far from complete - but is posted to initiate a conversation around if this direction feels like the right one to go in. The example in the |
Thanks for this great proposal @ld-kerley The proposed solution is elegant in the sense that it's language agnostic. By restructuring the ShaderNodes at ShaderGraph construction time it requires minimal changes to existing shader generators. But consequently it forces this solution on all existing and future languages, so a question is if that is ideal? For languages like OSL and MDL, that supports struct types, would we not want to have the MaterialX types mirrored in the generated code? The pros I can see is that maybe these languages, or future languages, can handle structs better internally than splitting them out to separate inputs. Also, from a shader interface point of view it feels more intuitive to get a shader out where the MaterialX types are represented as is, when structs are supported in the language. One alternative solution would be to have the struct represented as a nesting of ShaderInputs, where the struct ShaderInput remains but with child inputs added for each member. Then each generator can choose how to handle it: either generate a single struct input or split into member inputs. General question: I assume there are no plans to support connections to/from individual struct members? Instead we could have |
I am curious for thoughts on how integrations which do not use code generation parse these and handle interop? For instance, I'm working on the Khronos glTF representation of graphs and there is no way to represent a struct there, or even any notion of type definitions. It's "simple" types likes floats, ints and tuples / matrices of these. Things like mixing in string types, inheritance are also issues since they require run-time evaluation. Another connection point is naturally OpenUSD and how struct interop would work there? Throwing out the idea to move the translation ("flatten") of structs into non-structs outside of code generation. |
@niklasharrysson - I would propose we explicitly disallow connections to/from individual struct members. Having been a shader author myself in a past life, there have certainly been times when I have wanted to ensure my struct stays "whole", ie. isn't messed around with my someone else. And given we already have the I can speak for OSL that there would be no performance penalty for this approach, as OSL would just perform a very similar operation on the shaders in the shader group when it's constructed. "A struct in OSL is just syntactic sugar" - Larry Gritz). I'm not familiar with MDL, but I would observe that this proposal for the shader generation is doing nothing more than a shader author 'could' very reasonably do anyway. I do like the idea of in the shader generator space there being a higher level object that the specific language shader generators could specialize on, so that if MDL really did want to retain the struct as a first class connection point it could. The idea with the code here was really just to demonstrate the idea. I guess we could provide something similar to what I have here (albeit cleaned up), as a default implementation in MaterialXGenShader, and then specialize from there. @kwokcb - I think if an application/platform has decided to forgo using the shader generation, then they have decided to take on the burden of implementing the backend themselves. Then so long as we adequately specify what the expectations are, those platforms would be free to select whatever implementation suited them, but I don't think MateiralX would have any sort of other responsibility there. We could certainly outline in the specification this namespace parameter approach as a guidance. As for OpenUSD, as far as I'm aware at the moment there is no sort of formal specification of the formatted data that I think we 'could' move the flattening process somewhere earlier, potentially as an optional stage, but it feels a little like pushing the peas around the plate. I would assume that you'd end up in a similar situation of having to effectively author new |
Fully agree on disallowing connections to individual struct members, and the extract/combine paradigm can be used selectively, to keep it clean and simple. Yes I think it would be valuable to preserve the higher order structure in the ShaderGraph space, to let specific languages specialize on this as needed. We can have a default implementation that flattens this, and languages like MDL and OSL can override as needed. Regardless of performance, I like the way a generated shader's parameterization will map 1-1 to the MaterialX parameterization when structs are supported in the language. You can avoid the extra work of parsing parameter prefixes when interfacing with the shader. Two alternatives I can see for the higher order structure:
For (2) our current TypeDesc system is quite rigid and I think it would need a larger rewrite to support this. I'm working on changes to this system now anyway, but it's moving in the opposite direction there TypeDesc objects will become more lightweight. I think maybe option (1) is simpler and less intrusive. One complication to work out is how to handle struct |
@ld-kerley We've now merged development work on MaterialX 1.39 back to the |
Let's close out this pull request in favor of #1831 for now, and we can borrow ideas from this original pull request if needed in the future. |
Struct support in MaterialX
Introduction
The MaterialX (1.38) specification document allows for “Custom Data Types” (see page 8). Custom data types can specify
<member>
elements, effectively defining a custom structure. Once defined these custom structs can be used as types forinput
andoutput
elements in a node or node definition in MaterialX documents.Parsing the document is currently supported, but nothing else is. None of the MaterialX shader generation code provides any support for these custom types.
Proposal
Inspired by how OpenShadingLanguage deals with structs, I believe there is a reasonably easy path to adding support to MaterialX shader generation, in a language agnostic way. (Previously the project struggled to find a way to peformantly support this idea across all destination shader languages).
Effectively the idea boils down to implictly namespaced shader parameters being generated dynamically at shader generation time. Instead of a node generating a function where the parameters are one-to-one with the input ports on the corresponding node definition, when we encounter a custom struct type we expand that parameter out to a set of parameters prefixed with the struct name.
For example, the following custom type and node definition
would generate something similar to the following psuedo-code, note the namespaced input parameters to the function. Note, the separator string here (
__I__
) is certainly a subject for discussion.The shader generation code can be updated in a shader language agnostic way. Each time a struct is discovered when traversing the nodegraph each operation (connecting nodes, setting local values, etc) is just applied for each child
<member>
of the custom struct.Rationale
This might not be the most obvious approach, but the reason we choose to take it anyway, is primarily that it allows the nodegraph to give the illusion of using structs, without imposing any potential performance penalty on the destination shader language by actually using structs. It also allows a potential future shader language to be adopted if it doesn’t support the concept at all. There is strong precedent for this approach in OSL, which is a production proven shading language used on 100’s of feature films.
Considerations
A few subtleties that are open for discussion.
<member>
elements contain adefaultvalue
element to ensure nothing is ever uninitialized?Proposed Specification
I’ve written what I would like to propose to the ASWF project as the update to the MaterialX specification for 1.39.
Custom Data Types
In addition to the standard data types, MaterialX supports the specification of custom data types for the inputs and outputs of shaders and custom nodes. This allows documents to describe data streams of any complex type an application may require; examples might include spectral color samples or compound geometric data. The structure of a custom type's contents is described using a number of
<member>
elements.Types can be declared to have a specific semantic, which can be used to determine how values of that type should be interpreted, and how nodes outputting that type can be connected. Currently, MaterialX defines three semantics:
color
": the type is interpreted to represent or contain a color, and thus should be color-managed as described in the Color Spaces and Color Management Systems section.shader
": the type is interpreted as a shader output type; nodes or nodegraphs which output a type with a "shader" semantic can be used to define a shader-type node, which can be connected to inputs of "material"-type nodes.material
": the type is interpreted as a material output type; nodes or nodegraphs which output a type with a "material" semantic can be referenced by a in a .Types not defined with a specific semantic are assumed to have semantic="default".
Custom types are defined using the
<typedef>
element:Attributes for
<typedef>
elements:name
(string, required): the name of this type. Cannot be the same as a built-in MaterialX type.semantic
(string, optional): the semantic for this type (see above); the default semantic is "default".context
(string, optional): a semantic-specific context in which this type should be applied. For "shader" semantic types,context
defines the rendering context in which the shader output is interpreted; please see the Shader Nodes section for details.inherit
(string, optional): the name of another type that this type inherits from, which can be either a built-in type or a custom type. Applications that do not have a definition for this type can use the inherited type as a "fallback" type.hint
(string, optional): A hint to help those creating code generators understand how the type might be defined. The following hints for typedefs are currently defined:Attributes for
<member>
elements:name
(string, required): the name of the member variable.type
(string, required): the type of the member variable; can be any built-in MaterialX type; using custom types for<member>
types is not supported.defaultvalue
(string, optional): the value the member variable will be initialized to if not explicitly set on a node.If a number of
<member>
elements are provided, then a MaterialX file can specify a value for that type any place it is used, using an aggregated initializer, a semicolon-separated list of numbers and strings surrounded by braced. The expectation is that the numbers and strings between semicolons exactly line up with the expected <member> types in order. For example, if the following <typedef> was declared:Then a permissible input declaration in a custom node using that type could be:
Once a custom type is defined by a
<typedef>
, it can then be used in any MaterialX element that allows "any MaterialX type"; the list of MaterialX types is effectively expanded to include the new custom type.The standard MaterialX distribution includes definitions for four "shader"-semantic data types: surfaceshader, displacementshader, volumeshader, and lightshader. These types are discussed in more detail in the Shader Nodes.