Pass --cfg doc to check builds of dependencies as part of cargo doc/rustdoc

[djrenren]

Problem
Some packages such as llvm-sys have workarounds that allow their docs to build on docs.rs despite the fact that no usable binary can be generated (in this case because it relies on a platform binary that is not present on docs.rs). Essentially it uses detection of #[cfg(doc)] to allow builds to succeed when they would otherwise fail. This works well today.

However, when this project is a dependency in another project, cargo doc will not just run rustdoc, it will also invoke rustc to perform a check build. This bypasses any #[cfg(doc)] protections because it runs without --cfg doc. It is currently not possible to detect that your crate is being compiled in check mode as part of a larger doc run. The desired build characteristics, however, are the same. We don't need a working binary, and we're invoking the compiler for the purpose of docs.

It's also very strange to have a case where cargo doc works on a project but not when that project is a dependency.

I've produced a minimal example of this that demonstrates the issue without all the cruft of thinking about LLVM and system dependencies.

Steps

1. Clone djrenren/cargo-doc-failure
2. cd app
3. cargo rustdoc

Notice this fails while cargo rustdoc in the dependency succeeds.

Current Workarounds
Local workaround:
$ RUSTFLAGS="--cfg doc" cargo doc

Docs.rs workaround:
Cargo.toml:

[package.metadata.docs.rs]
rustc-args = [ "--cfg" , "doc"]

Suggested Solution(s)

1. As suggested in the title of the issue, pass the doc config flag in while doing check builds as part of cargo rustdoc

2. Provide a standard way to add cfg flags to libraries when built as part of rustdoc. That is, standardize what docs.rs already does so that it works locally as well.

Notes
This occurs on all cargo versions I've been able to find so the version is largely irrelevant, but let's say latest stable 1.47.0

Passing a doc flag by default to check builds could produce strange results if downstream crates use doc cfg's to change the interface of their crate, which may be a point in favor of solution 2.

[djrenren]

Also, I've marked this a bug, because of the unexpected behavior it produces, but I could see the argument to call it a feature request.

[jyn514]

See also rust-lang/rust#76252.

[jyn514]

Note that crates have a tendency to do very cursed things when cfg(doc) is passed (https://doc.rust-lang.org/nightly/rustdoc/advanced-features.html#interactions-between-platform-specific-docs, rust-lang/rust#75100), so this will break crates in practice. I think the tradeoffs are worth it (since some crates will be broken either way), but any changes here should at least get a crater run.

See also the discussion in rust-lang/rust#77276 (comment).

[laplab]

My team also suffers from similar problem. We are using different rustc version and nightly releases for generating documentation, mainly because they are more feature-rich at the moment. However, that makes some static assertions false, which we would rather avoid making in the first place when generating docs.

@ehuss Would the Cargo team be open to accepting contributions on this topic? Or do you feel like there is more discussion needed?

[ehuss]

The team discussed this a bit, and I think there is still quite a bit of open design exploration needed.

Some thoughts and questions:

• Can this be distilled to just build.rs scripts? I see @laplab mentions they have static assertions that would have issues, but it's not clear what exactly those look like. What exactly are those static assertions? Can they be resolved in other ways?
• There is potentially a very large performance and caching cost. The cargo doc cache is currently shared with cargo check. Splitting those could incur a very significant increase in build times for some users, and a large increase in disk usage for something that may not normally be needed.
• There are other issues like Cargo doc does not pass --cfg doc for build scripts #8944 and Inform build scripts whether cargo is checking or building #4001 for figuring out how build scripts should behave for different scenarios like cargo check. I think I would echo the thoughts and questions in Inform build scripts whether cargo is checking or building #4001 (comment) about the viability and meaning of running build scripts in different modes.
• We don't know the scope of the breakage this could have. It may be feasible for someone to do a crater run to get that data. If there is a large amount of breakage, then we would know for sure that a different approach would be needed.

[djrenren]

These are some fantastic thoughts. Thanks for taking the time to discuss. I hadn't thought of the caching problem and it is indeed super important to keep these caches working.

It gets me thinking that maybe this problem is better solved in the compiler. The llvm-sys example above delays error messages to post-build script, by setting an environment variable and putting the following code into the crate:

#[cfg(all(not(doc),LLVM_SYS_NOT_FOUND))]
std::compile_error!(/*...*/)

This approach runs into problems because there's no cfg flag for check builds, but if we imagine there was, then we could extend the llvm-sys approach to solve at least the problem that initiated this issue.