skeptic setup

[mockersf]

integration with skeptic to validate book!

Everything for the integration seems to work fine (Zola not bothered by skeptic annotations, skeptic loading templates). Two notes:

• I excluded the migration guides folder as it will contain invalid code
• skeptic want a language of rust, not rs

that said, it fails on my computer because:

error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-m64" "-arch" "x86_64" "-L" [blablablabla]
  = note: ld: library not found for -lglslang.glsltospirv
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

not sure how to move forward on that

[mockersf]

in ci:

error: test failed, to rerun pass '--test skeptic'
  = note: /usr/bin/ld: cannot find -lglslang.glsltospirv
          /usr/bin/ld: cannot find -lHLSL.glsltospirv
          /usr/bin/ld: cannot find -lOGLCompiler.glsltospirv
          /usr/bin/ld: cannot find -lOSDependent.glsltospirv
          /usr/bin/ld: cannot find -lSPIRV.glsltospirv
          /usr/bin/ld: cannot find -lSPVRemapper.glsltospirv
          /usr/bin/ld: cannot find -lSPIRV-Tools-opt.glsltospirv
          /usr/bin/ld: cannot find -lSPIRV-Tools.glsltospirv
          collect2: error: ld returned 1 exit status

[mockersf]

Build failure is probably due to budziq/rust-skeptic#104 and our good old bevy_glsl_to_spirv.

This means that we can already test all code snippets that don't use bevy_render which should be a lot.

To be able to document everything we could either:

• Fix skeptic
• Switch to naga
• Build and test on an arch that needs shaderc and available on GitHub actions
• Add features in Bevy to switch to shaderc not only on arch

[mockersf]

Added a few more tests. Writing the skeptic template is not fun but not hard either

Report is ok.
Success:

     Running tests/skeptic.rs (target/debug/deps/skeptic-d7b44fede4443f9e)

running 11 tests
test index_sect_bevy_ecs_line_19 ... ok
test index_sect_your_first_components_line_68 ... ok
test index_sect_your_first_components_line_74 ... ok
test index_sect_your_first_components_line_101 ... ok
test index_sect_bevy_ecs_line_31 ... ok
test index_sect_bevy_ecs_line_23 ... ok
test index_sect_your_first_components_line_115 ... ok
test index_sect_your_first_components_line_80 ... ok
test index_sect_your_first_system_line_41 ... ok
test index_sect_your_first_components_line_90 ... ok
test index_sect_your_first_system_line_49 ... ok

test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.68s

Failure (I added an error):

     Running tests/skeptic.rs (target/debug/deps/skeptic-d7b44fede4443f9e)

running 11 tests
test index_sect_your_first_components_line_101 ... FAILED
test index_sect_bevy_ecs_line_19 ... ok
test index_sect_your_first_components_line_74 ... ok
test index_sect_bevy_ecs_line_31 ... ok
test index_sect_your_first_components_line_68 ... ok
test index_sect_bevy_ecs_line_23 ... ok
test index_sect_your_first_components_line_80 ... ok
test index_sect_your_first_components_line_115 ... ok
test index_sect_your_first_system_line_41 ... ok
test index_sect_your_first_components_line_90 ... ok
test index_sect_your_first_system_line_49 ... ok

failures:

---- index_sect_your_first_components_line_101 stdout ----
error[E0277]: the trait bound `Name: WorldQuery` is not satisfied
  --> /var/folders/y2/6ll2wvjd5q3fssp45tx5nf3h0000gn/T/rust-skepticSu8jWW/test.rs:8:28
   |
8  |     fn greet_people(query: Query<Name, With<Person>>) {
   |                            ^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `WorldQuery` is not implemented for `Name`
   |
  ::: /Users/francois/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/bevy_ecs-0.5.0/src/system/query.rs:14:25
   |
14 | pub struct Query<'w, Q: WorldQuery, F: WorldQuery = ()>
   |                         ---------- required by this bound in `bevy::prelude::Query`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0277`.
thread 'index_sect_your_first_components_line_101' panicked at 'Command failed:
"rustc" "/var/folders/y2/6ll2wvjd5q3fssp45tx5nf3h0000gn/T/rust-skepticSu8jWW/test.rs" "--verbose" "--crate-type=bin" "--edition=2018" "-L" "/Users/francois/Dev/vleue/bevy_friends/bevy-website/book-validation/target/debug" "-L" "/Users/francois/Dev/vleue/bevy_friends/bevy-website/book-validation/target/debug/deps" "--target" "x86_64-apple-darwin" "--extern" "bevy=/Users/francois/Dev/vleue/bevy_friends/bevy-website/book-validation/target/debug/deps/libbevy-517749bc23e48943.rlib" "--extern" "skeptic=/Users/francois/Dev/vleue/bevy_friends/bevy-website/book-validation/target/debug/deps/libskeptic-df5afd70542a72c9.rlib" "-o" "/var/folders/y2/6ll2wvjd5q3fssp45tx5nf3h0000gn/T/rust-skepticSu8jWW/out.exe"', /Users/francois/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/skeptic-0.13.6/src/rt.rs:127:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    index_sect_your_first_components_line_101

test result: FAILED. 10 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.09s

It may get hard to identify source of failure if there are a lot.

[cart]

So I think in summary:

1. skeptic works in the context of zola!
2. debugging skeptic failures will get more challenging as the number of snippets increases
3. native deps break skeptic. examples that require native deps should probably just use the file_code_block method. hopefully bevy can move away from native deps for shader compilation, but even if we do, theres no guarantee that we won't add more native deps in the future (ex: distill uses a database with c bindings).
4. skeptic templates are a bit cumbersome and most snippets will need custom templates to compile. is this worth the complexity cost / skeptic quirks vs using file_code_block?

I think if we can solve (3) and (4) skeptic is probably "worth it" from my perspective. If we can use the same 1-2 templates for all examples (ex: by supporting "hidden code" in code blocks that skeptic executes but zola hides), that solves (4). Idk the best way to handle (3) short of committing to never use native dependencies (which feels hard to me). I wish skeptic supported "crate templates" that use cargo to compile the examples.

Now that you have hands on experience, what would you recommend (both in the short and long term)? In the short term I'm sort of leaning toward "add anchor support to file_code_block and just use that exclusively". It makes the debugging situation easier, supports native dependencies (and therefore present and future versions of bevy), and the "most examples require custom skeptic templates" problem means that we're already paying the "external file complexity cost" with skeptic.

[mockersf]

I'm very biased towards file_code_block...

That said, including a block from a file that can be built by itself will always cover more cases. Snippets in doc are nice for small code snippets, but having to keep two different build systems to handle both is probably not a good idea. Having to write the template negates the simplicity of just writing the snippet in the .md file.

It could get better if we get both Zola and skeptic to handle hidden lines (or maybe doable only in Zola: code would be close to what existing hl_lines is doing). Or something like budziq/rust-skeptic#77 (all the code blocks from a .md file in a single test) but that PR hasn't moved in 2.5 years.

Short term, file_code_block handles all our needs (and would be perfect for a step by step tutorial like #102).
Long term, handling both could be nice if we reduce complexity of skeptic case to offset the need of two build systems.

[mockersf]

opened a discussion on Zola for hiding lines https://zola.discourse.group/t/hide-lines-from-code-block/898

[mockersf]

getzola/zola#1453 has been merged which would allow us to use less templates with skeptic in next Zola version

[mockersf]

This should be revisited without skeptic:

• have an empty crate that just include as doc every markdown file with code blocks (possible since Rust 1.54)
• hide setup lines from code blocks (possible since Zola 0.14)

[alice-i-cecile]

Closing in favor of #150.