Compiling Shaders

[questor]

Hi,
I like your implementation and thought about using it to convert my lightprepass renderer to your framework (and release the source of course). Sorry for the long text...

Beside missing multiple render targets (which should be easy to add) I've had in my own implementation some ideas that aren't possible with your implementation. For example in your shader compiler you write the text of glsl shaders to the bin-file, but you precompile the hlsl-shaders for dx9/dx11. In my opinion it would be much better to write in the dx9/dx11 case also only the pure text and let your framework compile the shader during loading (and perhaps let the app ask for the compiled binary blob so that the app can do it's own cache management).
Rational behind this: if you compile the shader during createVertexShader/createFragmentShader you can specify special defines so that a shader can work with different vertex formats. To get the whole picture here's an example:
first I create a vertex format via templating (has the advantage that the ide knows about the components of a vertex and can provide autocompletion):
typedef VERTEXBUILDER_3(POS_3F, NORMAL_3F, TEXCOORD0_2F) ReconstructVertexFormat;
now when I compile my shaders I provide also the vertexbuilder format to the compiler subsystem which inserts automatically defines for the vertex format so that I can write in the shader the following:
struct ReconstructVertexShaderInput {
float3 Position : POSITION0;
float2 TexCoord : TEXCOORD0;

ifdef VERTEXCOMP_COLOR_4U

float4 Color : COLOR;

endif

};
or can customize which light model to use or something like that...

I can provide you with my implementation of the vertexbuilder if you want to take a look...

another thing would be the possibility to scan for predefined values like worldviewprojection or something like that and set only the values really needed by the shader and not simply all values for every shader.

[bkaradzic]

Yes GLSL is text in .bin file, but technically they should be considered binary (they are only in source form because that's how GL expect it, if I could use binaries there I would :). Source GLSL shader passes throught GLSL optimizer and it outputs optimized version of GLSL. HLSL has binary representation. The reason why I don't pass HLSL source is because I don't want to link or dynamically load D3DX DLL. You should be able to pass different vertex attributes to shaders, even if they mismatch shader inputs.

As for multiple render targets, I'm planning to add them, but I want solution that would work for GLES2, so that part is a bit more difficult. Anyhow it's on my todo list to be done soon.

[questor]

okay, makes sense somehow :) It also wouldn't be very clever to optimise something still with compile switches... perhaps I have to think more about a shader configuration (a json file with all possible define combinations that are allowed on a shader and the compiler compiles all combinations). but that would be something for a upper software layer, not for bgfx itself...

[bkaradzic]

Yeah idea is to keep those shader code generators in layer above bgfx. shaderc is running full preprocessor, so if you can achieve what you need with some preprocessor magic, you could do it with shaderc. Also you could generate multiple .sc files in some scripting language, and then run it through shaderc to get final binaries. If you share small example on Gist I might have better idea what you're trying to achieve and I might have solution for it.