GPU-enabled CI tests for Bevy

[alice-i-cecile]

What problem does this solve or what need does it fill?

As we work on the renderer, we have no way to detect GPU-only failures in changes made to Bevy.

What solution would you like?

We also run our CI with a GPU.

What alternative(s) have you considered?

Authors manually run the GPU examples just before merging in GPU-related code.

CPU-simulated GPUs are apparently another choice.

Additional context

This wgpu CI looks like interesting prior art. See #1706 for a particularly nasty example of this failure mode.

[mockersf]

Sadly a GPU often costs money...

Do we want to

1. run tests on GPU on every commit in a PR?
2. run tests on GPU only in Bors merge branch?
3. run tests on GPU only on commit to main?

If we want to reduce/control costs, I think 2. is a good start. 1. means the risk of being "under attack" as someone malevolent could push commits in a PR just to ramp up the bill. 3. is too late as the commit has been merged.

From GitHub insight, there was 88 commits to main in the last month. If we take 20 minutes to run all examples (build time + run time), that means something like 30 hours per month (rounding up).

If we look at the pricing of a company like CleverCloud, at 0.50$ per hour, that gives us 15$ per month.

I'm talking about CleverCloud as it's a French company that I know (but have no parts in), they have a very good reputation, they love rust, and they also have an open source project sponsoring policy (bottom of https://www.clever-cloud.com/en/open-source). And their support is great.

Would that be something interesting to pursue?

[alice-i-cecile]

We could also further restrict it to code that makes non-docs changes to bevy_render and friends, which would reduce the cost.

I think pursuing open source sponsorship for this is an excellent idea: we're a compelling project with very modest needs. @cart do you think this is a good idea? Is it okay if community members pursue sponsorships of Bevy like this?

[cart]

I'm totally cool with people pursuing sponsorships on behalf of the project, provided they check with me first. In this case, I think it would be good to have some sort of useful gpu test set up prior to asking for their sponsorship, so we don't waste their time (or our "sponsors page space") on something we aren't sure we need yet.

[archmonduu]

If you base your docker container off of utensils/opengl:stable you can get bevy 0.4 to work even in tests that have graphics. At least it looks like it, my pipeline is green where it used to panic on no device found.

hacky Dockerfile for reference

FROM utensils/opengl:stable

ENV CARGO_HOME=/opt/cargo

ENV PATH=$CARGO_HOME/bin:$PATH

RUN apk update
RUN apk add curl
RUN curl https://sh.rustup.rs -sSf | sh -s -- -y  --default-toolchain none

RUN rustup toolchain add stable beta nightly

RUN rustup component add --toolchain nightly rustfmt-preview

RUN rustup component add --toolchain nightly clippy-preview

# shaderc dependencies
RUN apk add build-base cmake git python3 && apk upgrade

# To have dependencies already fetched in the image - improves runtimes in CI later.
COPY Cargo.toml ./
RUN mkdir src && touch src/main.rs && touch txt.txt
RUN rm -rf ~/.cargo
RUN cargo fetch --verbose

# Patch bevy to build on alpine... (with shaderc)
ADD ci_utils ./
RUN cp shader.rs $(find -name shader.rs | grep bevy_render)
RUN cp render_cargo.toml $(find -name Cargo.toml | grep bevy_render)
RUN cp build.rs $(find -name build.rs | grep glsl-to-spirv)

Utility files in https://gitlab.com/torlachnil/bevy-project/-/tree/master/ci_utils (ignore the rest of the project please :] ), done to get around #1348

[qarmin]

From my testing looks that SwiftShader(similar to lavapipe from mesa which allow to emulate Vulkan on CPU) works fine in CI with Bevy.
You can take how to setup it from - https://github.com/godotengine/godot/blob/3e710bb276bd48e34ea7ce696b7948f4ba607435/.github/workflows/linux_builds.yml#L143-L164

Steps are really simple:

1. Download precompiled SwiftShader libraries(e.g. https://github.com/qarmin/gtk_library_store/releases/download/3.24.0/swiftshader.zip) or compile it manually - instruction Cannot run Godot on virtual X desktop with xvfb-run and lavapipe godotengine/godot#38428 (comment)
2. Compile test bevy project
3. Extract/Copy SwiftShader .so library and fill json file as in instruction from above(path inside json must be absolute)
4. Execute command VK_ICD_FILENAMES=./vk_swiftshader_icd.json ./bevy_project

In Godot we use special project which runs ~40 seconds and it is used mostly to catch crashes, invalid memory accesses, memory leaks etc.
In Bevy I see that in 381 places is used unsafe word, so I think that this project also could find such errors even if it is written in Rust and Godot in C++.

[mockersf]

thanks @qarmin, this was easy to follow and setup, and now part of our CI too 🎉

[alice-i-cecile]

I'm closing this for now following #1826. We can open up some new issues for more advanced needs that arise :)