Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cargo test doesn't compile doc tests for target != host #6460

Closed
therealprof opened this issue Dec 18, 2018 · 6 comments · Fixed by #6892
Closed

cargo test doesn't compile doc tests for target != host #6460

therealprof opened this issue Dec 18, 2018 · 6 comments · Fixed by #6892
Labels
C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` Command-test

Comments

@therealprof
Copy link

Problem

We'd like to have no_run doc tests for embedded HAL impl crates to ensure that the crate is sufficiently and accurately documented. However cargo doc completely ignores those tests when running with the --doc flag:

# cargo test  --release  --features="stm32f042,rt" --doc
    Finished release [optimized + debuginfo] target(s) in 0.07s

while creating an error when trying to run all tests:

# cargo test  --release  --features="stm32f042,rt"
   Compiling stm32f0xx-hal v0.9.0 (/Users/egger/OSS/stm32f0xx-hal)
error[E0463]: can't find crate for `test`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `stm32f0xx-hal`.

I would expect that no_run tests are correctly compiled in the current environment (as are docs, libraries, executables and examples) and compile errors flagged as test failures. In the future it would be even nicer if I could define a test runner environment (like qemu or docker) to test the executables as well.

Steps

  1. Obtain https://github.com/stm32-rs/stm32f0xx-hal
  2. cargo test --release --features="stm32f042,rt" --doc
  3. cargo test --release --features="stm32f042,rt"

Possible Solution(s)
Attempt to compile doctests even when target != host, especially no_run ones.

Notes

Output of cargo version:
cargo 1.33.0-nightly (2cf1f5d 2018-12-11)
cargo 1.31.0 (339d9f9 2018-11-16)

@ehuss
Copy link
Contributor

ehuss commented Dec 18, 2018

I think this is intentional, see rust-lang/rust#31907. I wonder, how much risk there is that doc tests will fail to just compile on host?

See also rust-lang/rust#44404 for a request to have rustdoc support --target.

@alexcrichton
Copy link
Member

Yes this is currently intentional that we don't run rustdoc at all, but it makes sense to me that we could have an option to try to at least compile examples even if we can't run them!

I suspect that we can't do this in a backwards compatible fashion by default, but at least having an option in the maniest for something like this seems reasonable to me

@alexcrichton alexcrichton added Command-test C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` labels Dec 19, 2018
@therealprof
Copy link
Author

@alexcrichton I fear that the inability to compile doc tests will have a major (negative) quality impact for embedded because there's no other way to build software but via cross-compilation. Thus it would be great if this feature would be given appropriate consideration and priority.

@ririsoft
Copy link

I fear that the inability to compile doc tests will have a major (negative) quality impact for embedded because there's no other way to build software but via cross-compilation. Thus it would be great if this feature would be given appropriate consideration and priority.

This also impacts developers using pc-windows-gnu target on a system where pc-windows-msvc is the default. I discovered a bug in my documentation thanks to a CI build running on linux, but I wish I discovered this earlier in my pipeline.

@jethrogb
Copy link
Contributor

At the very least, I think cargo should error if you explicitly pass --doc and --target.

bors added a commit to rust-lang/rust that referenced this issue Sep 10, 2019
Allow cross-compiling doctests

This PR allows doctest to receive a --runtool argument, as well as possibly many --runtool-arg arguments, which are then used to run cross compiled doctests.
Also, functionality has been added to rustdoc to allow it to skip testing doctests on a per-target basis, in the same way that compiletest does it. For example, tagging the doctest with "ignore-sgx" disables testing on any targets that contain "sgx". A plain "ignore" still skips testing on all targets.

See [here](rust-lang/cargo#6892) for the companion PR in the cargo project that extends functionality in Cargo so that it passes the appropriate parameters to rustdoc when cross compiling and testing doctests.

Part of [#6460](rust-lang/cargo#6460)
bors added a commit that referenced this issue Sep 18, 2019
Added ability to crosscompile doctests

This commit adds the ability to cross-compile and run doctests.
Like before cargo checks if target == host, the difference is that if there is a runtool defined in config.toml, it passes the information forward to rustdoc so that it can run the doctests with that tool. If no tool is defined and the target != host, cargo instead displays a message that doctests will not be compiled because of the missing runtool.

See [here](rust-lang/rust#60387) for the companion PR in the rust project that modifies rustdoc to accept the relevant options as well as allow ignoring doctests on a per target level.
Partially resolves [#6460](#6460)

See [here](#7040) for the tracking issue.
@therealprof
Copy link
Author

Woohoo!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` Command-test
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants