Javelin post

[kvark]

Rendered
cc @antiagainst

[msiglreith]

its Metal backend

[Lokathor]

SPIRV-Cross may be the "last bastion of C++ code in gfx-rs" but as far as I recall there's still not a pure rust way to make SPIRV from shader source text, right?

Will Javelin allow translating source text into SPIRV as well? Your post isn't clear on this point.

[kvark]

@Lokathor

there's still not a pure rust way to make SPIRV from shader source text, right?

There is no single SPIR-V source text, unless you mean the textual form (slide 38), which nobody uses.

Some people generate SPIR-V from GLSL, most use HLSL as the source. Rust ecosystem also dips the toes in the direct SPIRV generation with rlsl. We can't cover all of them here, and we don't think it makes sense to even try covering that part in scope of Javelin. It's an important path to get right, but it's not essential for gfx-rs to operate (since it only knows and cares about SPIR-V).

I'll make the article more clear about this (thank you for noticing this!)

[Lokathor]

Right, I mean GLSL -> SPIRV or HLSL -> SPIRV or any other human readable form to SPIRV.

I don't think that gfx-rs needs to cover every format, but I think that it needs to cover at least one format. Otherwise, a person cannot legitimately say that you can use just gfx-rs to display things, because you have no way (without reaching outside of gfx-rs) to actually get SPIRV code to send to the GPU.

[kvark]

@Lokathor I think you have a valid point, but I'm not sure about the best way to address it. Do you suggest to include Something -> SPIRV in scope for the project? I'd very much prefer to not fix the target on GLSL or HLSL and instead keep exploring.

[Lokathor]

I don't know enough to say what the best shader situation would be. I started graphics with OpenGL, so then when I did learn-gfx-hal I also naturally used GLSL for the basic shaders I needed there.

Someone smarter than me will have to pick a first target for supporting a pure-rust FOO->SPIRV compiler. GLSL, HLSL, RLSL, or even some fourth option. Keeping in mind that it wouldn't be a total lock in, just a first selection and that probably other compilers for the other formats would follow.

[antiagainst]

Thanks for the interest in rspirv! It's thrilling to see that it can help to remove the last C++ code component in gfx-rs! I'm happy to facilitate the procedure. :)

I'm not familiar with gfx-rs per-se but I think HLSL is the front-end language chosen by the industry. Developing a compiler for HLSL to SPIR-V in Rust is a non-trivial task given that HLSL is a C++-ish language. Designing another Rust DSL for shading language might be even daunting. I think they are both significant enough to merit their own project. But this is just my two cents. :)

[Lokathor]

I think that if we have a reliable, pure-rust "SourceLang -> SPIRV -> NativeLang" pipeline it doesn't really matter that much which language is the textual source language. They're basically doing the same pure math ops since they're all fairly plain languages running on the same GPU, it's just syntactic differences.

If we setup an HLSL compiler to start and then say "Sorry that you can't use GLSL with this, here's a 30 minute tutorial to get you up to speed on how to use HLSL instead, and here's some cross compilers so that you can easily translate your old GLSL to HLSL as you move forward." then that's a fine situation that doesn't really leave anyone stuck.

[fu5ha]

Nice :) I just have some gramatical/sentence flow things

[fu5ha]

We are having fun moving the gfx-rs project forward thanks to the power provided by the Rust language.

We are building rich abstractions which expand beyond the gfx-hal API itself and into the internal layers of the backends, structured to be modular and maintainable.

We do all of this while requiring only a portion of our developers' time and attention, who work on gfx-rs mostly as a side project.

[kvark]

do you think the other sentences there are best to be removed?

[fu5ha]

No, just only had comments on those ones :)

[fu5ha]

That is what the Javelin project is about...

It's a very complex piece of software and we have not made much progress on it yet.

[fu5ha]

The key advantages we hope to gain by implementing Javelin are:

[grovesNL]

@Lokathor I don't think gfx needs to take much of a stance on HLSL/GLSL/RLSL -> SPIR-V compilation at the moment, besides agreeing that it would be nice to have some fully Rust tooling in that space. gfx-hal (like Vulkan) just accepts SPIR-V and it isn't impacted by the choice of source language used.

For example, somebody could contribute to the SPIR-V backend for the existing glsl crate or contribute to RLSL if they would like to help.

[grovesNL]

as land the production code

Maybe "land production-quality code"

[grovesNL]

We could also mention how we have to have a separate Emscripten build to generate wasm for SPIRV-Cross, separate from the rest of the Rust code (which does not use Emscripten)

[grovesNL]

It should be AIR instead of MSL

[grovesNL]

Maybe we could clarify it a bit more

SPIR-V to anything (for example, translating SPIR-V to HLSL, MSL, or GLSL)

[grovesNL]

Maybe we could rephrase

It is also written in a way (arguably!) that makes weak guarantees about what the state of the translator is at any point.

to talk about how we'd prefer to work with immutable Rust data structures representing the compiler state directly instead of going through the Rust <-> C ABI <-> C++

[grovesNL]

building the proper structured representation in rspirv

to

building a proper structured representation for SPIR-V in rspirv

[grovesNL]

We should probably clarify how a shader playground would relate to the existing efforts of http://shader-playground.timjones.io/

[Lokathor]

@grovesNL As I said in the Discord but I'll repeat here for others reading this thread: I absolutely do not care about the difference between GLSL/HLSL/anything-else. What I care about is that there is some way to produce the SPIRV that gets fed in to Javelin. If you're relying on a C/C++ tool for the start of the process, that entire process relies on C/C++. So Javelin doesn't really remove the last bit of C++ from the gfx-rs ecosystem.

The only reason that I talked about HLSL is because Antiagainst said that HLSL is used the most. That's fine if that's the first language that can be compiled to SPIRV, because basically all shader languages are the same anyway. The choice is unimportant compared to the fact that we can compile anything at all to SPIRV.

[kvark]

@Lokathor It's not nearly as black and white as you are painting it. Generation of SPIRV can be done with Rust code ("rlsl" and "glsl" crates), could be done using external tools, which don't have to be a part of the build process. At the very least, it could be done in a specially crafted environment that isn't required for the rest of the code. Consider the following scenarios:

A: I built a library on gfx-rs that relies on SPIR-Cross. Every single user of that library, on every platform, now has to be able to build SPIRV-Cross for their environment.

B: I built a library on gfx-rs that relies on Javelin. I grabbed some SPIRV shaders from the internet, maybe generated them with playground, or locally on one specific machine. I included them in the library - bam! no clients need to have any specific build environment.

---

There are many ways to generate SPIR-V. We do care about them. Technically, providing the trusted and idiomatic solutions for the ecosystem would be in scope for the gfx-rs organization. But it would not be in scope for this particular project - Javelin.

[Lokathor]

I understand everything you're saying, and I really fully understand that SPIRV can come from many locations and that some programs need to generate SPIRV at runtime and that most only need to generate it all compile time. I get all of that. I've used gfx-hal before. But everything I'm saying is just a strong, emotional reaction to the particular phrasing being used in the blog post:

We think the time has come for us to attack that last bastion of C++ code in gfx-rs. That is what the Javelin project is about:

Because SPIRV-cross isn't the last bastion of C++ for as long as there isn't a pure rust path at the start of the "ShaderText->SPIRV->Platform" pipeline.

• I looked at the rlsl crate the other day, it specifically says it's not ready for use, it doesn't accept contributions, and it's not on crates.io.
• The glsl crate is at least on crates.io but self rates as "passively maintained" and the documentation is totally unclear how you turn GLSL into SPIRV, if you can even actually do that at all (I actually don't think it can so that).

In my experience with trying to on-board beginners into gfx-hal, I've never once had any of them think twice about spirv_cross being a dependency. It's so seamless they don't even know it's in there. But I've had nearly every single one of them look at shaderc and its wild build process and super long build times and comment on how that is the most annoying and problematic step in the entire build.

Thank you everyone for your time.