Improve Testing and CI

[Meziu]

We should mass produce tests now that the API is getting stabilized, and integrate with the CI to make sure the examples build after changes.

[ian-h-chamberlain]

Definitely agree, unfortunately after doing a bit of research, I found out some things:

• Our current CI does not appear to actually check any doctests at all, so we will want to fix that. related cargo test --all-targets does not run doc tests rust-lang/cargo#6669
• Doc tests are... well, kind of an unfinished feature it seems like. A few gaps we will most likely run into:
  • We need an unstable flag to even compile doctests: Tracking issue for cross-compiling doc tests (doctest-xcompile) rust-lang/cargo#7040
  • --no-run is not supported by rustdoc / cargo:

     $ cargo 3ds test --doc -Zdoctest-xcompile
     No pre-build std found, using build-std
     error: Can't skip running doc tests with --no-run

    We can probably work around this in cargo-3ds somehow since this still seems to work: cargo test -vvv -Zdoctest-xcompile -Zbuild-std --target armv6k-nintendo-3ds --doc.
    See also rustdoc: Tracking issue for --no-run rust-lang/rust#87022
• For CI testing purposes, we can try using target.armv6k-nintendo-3ds.runner Cargo config but we don't really have a good way of verifying the tests actually pass. I have some ideas involving citra but it might take a bit of work

For now, we should probably keep all the examples as ```no_run, and try adding something to CI to make sure they at least compile. I might open a separate issue for the runner itself, since I think that will be a bit more involved and might even require changes to upstream citra to be effective.

[ian-h-chamberlain]

With lots of doctests in #134 I've marked this "In Progress", at least. I haven't had a chance to revisit the citra runner to really test everything though, so I'd say let's leave it open for now.

[Meziu]

I've noticed the CI still needs a little boost, and unit tests are very much lacking, so this issue needs to stay open for a while longer.

[ian-h-chamberlain]

I've made some progress on a proof of concept for running tests using Citra:
https://github.com/ian-h-chamberlain/test-runner-3ds

So far I can get it to work with integration + unit tests, but it's not running doctests yet.

Example of passing: https://github.com/ian-h-chamberlain/test-runner-3ds/actions/runs/5781441190/job/15666535932
and failing: https://github.com/ian-h-chamberlain/test-runner-3ds/actions/runs/5781347357/job/15666313158

For now, I just copied all the test runner code out of ctru-rs — if this seems like a good direction to continue in, I can migrate the test-runner-3ds repo to rust3ds/test-runner (open to better names too) and start the process of using it in ctru-rs.

I wanted to have this as a separate repo so we can reuse it for e.g. citro3d-rs and make it usable for others if they want. I don't think we strictly need to publish the crate either, since cargo allows for git dev-dependencies when publishing.

[adryzz]

I have recently made an app (in rust of course) to run unit tests back-to-back on physical consoles. it uses a reimplementation of 3dslink and the rosalina 3dsx loader, and almost works perfectly, just needs a few fixes for launch parameters.

It also supports toggling Luma3DS's GDB stub on a very hacked together branch.

This works when compiling the app as a .cia file to install, and a few additions to pass back the result to the app, to then log (both to a file and the bottom screen).

This is the end result, although this branch is very cursed and i'm adding things bit by bit as i clean them up