Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give precedence to html_root_url over --extern-html-root-url by default, but add a way to opt-in to the previous behavior #82776

Merged
merged 1 commit into from
Aug 21, 2021

Conversation

jyn514
Copy link
Member

@jyn514 jyn514 commented Mar 4, 2021

What is an HTML root url?

It tells rustdoc where it should link when documentation for a crate is
not available locally; for example, when a crate is a dependency of a
crate documented with cargo doc --no-deps.

What is the difference between html_root_url and --extern-html-root-url?

Both of these tell rustdoc what the HTML root should be set to.
doc(html_root_url) is set by the crate author, while
--extern-html-root-url is set by the person documenting the crate.
These are often different. For example, docs.rs uses
--extern-html-root-url https://docs.rs/crate-name/version to ensure
all crates have documentation, even if html_root_url is not set.
Conversely, crates such as Rocket set doc(html_root_url = "https://api.rocket.rs"), because they prefer users to view the
documentation on their own site.

Crates also set html_root_url to ensure they have
documentation when building locally when offline. This is unfortunate to
require, because it's more work from the library author. It also makes
it impossible to distinguish between crates that want to be viewed on a
different site (e.g. Rocket) and crates that just want documentation to
be visible offline at all (e.g. Tokio). I have authored a separate
change to the API guidelines to no longer recommend doing this:
rust-lang/api-guidelines#230.

Why change the default?

In the past, docs.rs has been the main user of --extern-html-root-url.
However, it's useful for other projects as well. In particular, Cargo
wants to pass it by default when running --no-deps
(rust-lang/cargo#8296).

Unfortunately, for these other use cases, the priority order is
inverted. They want to give precedence to the URL the crate picks, and
only fall back to the --extern-html-root if no html_root_url is
present. That allows passing --extern-html-root unconditionally,
without having to parse the source code to see what attributes are
present.

For docs.rs, however, we still want to keep the old behavior, so that
all links on docs.rs stay on the site.

@jyn514 jyn514 added T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. needs-fcp This change is insta-stable, so needs a completed FCP to proceed. requires-nightly This issue requires a nightly compiler in some way. labels Mar 4, 2021
@rust-highfive
Copy link
Collaborator

r? @GuillaumeGomez

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 4, 2021
@Nemo157
Copy link
Member

Nemo157 commented Mar 4, 2021

docs.rs will now link some dependencies to external sites instead of docs.rs.

This feels like a bad change to me. A large part of the value I see in docs.rs is that it's a one-stop-location for all rustdoc output of crates.io crates. A couple of the first reasons I can think of:

  • While rustdoc's HTML is usable without the JS enhancements, it's not the greatest experience, and I consider docs.rs a more trustworthy site to enable JS on than some random project's site.

  • A lot of self-hosted documentation sites disappear over time from bitrot. While documentation can already include external links that go dead, having at the very least the rendered rustdoc available gives a decent baseline of documentation for all crates.

  • It ensures a minimum of testing of generating the documentation on machines other than the developers. If the developers hosted documentation was used instead the first place any portability issues with building the documentation might be encountered would be one of their users attempting to build local docs. Currently there's a decent chance that if docs.rs fails to build the documentation the developers will notice this and investigate why. (Though docs.rs is more limited than users in some ways, like the lack of network access, but some might consider that a good thing to try and make the developers question why they are accessing the network in a build script 😁).

@dtolnay
Copy link
Member

dtolnay commented Mar 4, 2021

The rationale mentions Buck but I need to call out that this change is not so great for Buck. 😦

Generally how the Buck-based (and Bazel too) Rust build workflow works is third party dependencies are turned into build targets via a tool like https://github.com/facebookincubator/reindeer or https://github.com/google/cargo-raze, first party targets are handwritten depending on them just like you would write a Cargo.toml, and then the build tool directly invokes rustc and rustdoc according to the information present in the build targets. See e.g. https://github.com/dtolnay/cxx/blob/3dafaede930026394649639ac0798aa39d6b439f/BUCK#L1-L9 for an example of what this stuff looks like.

When it comes to rustdoc and third party open source libraries in the crates.io / docs.rs ecosystem, the behavior we need to accomplish is that third party libraries link to their preferred html_root_url if set, otherwise to docs.rs if not set, and first party code links to our own builds of the documentation hosted on private infrastructure.

The reason this change is not so good for Buck is that it requires targets downstream to heuristically figure out which of their upstream dependencies are third vs first party in order to construct an appropriate rustdoc invocation. Like if I have internal_crate1 which depends on serde + internal_crate2, then the rustdoc invocation corresponding to internal_crate1's docs build must figure out to pass --extern-html-root-url serde=docs.rs/... and --extern-html-root-url internal_crate2=docs.local/.... Reliably distinguishing which targets are first party or third party is complicated in a way that generalizes over the ways different monorepos make use of Buck. For example some heuristic like "if the top-level directory is named third-party" is brittle.

For supporting Buck's use case, a better approach from rustc and rustdoc would be to accept a fallback html-root-url as part of the downstream crate's build, with lower precedence than an html_root_url attribute set by the same crate. Then we'd have reindeer and cargo-raze emit third-party targets that look like (post Buck macro expansion):

rust_library(
    name = "serde",
    version = "1.0.123",
    srcs = [...],
    rustc_flags = ["--html-root-url=https://docs.rs/serde/1.0.123/serde"],
)

and similarly a url for the internal first party docs builds filled in by Buck macros for first party crates.

This accomplishes all of the objectives. Third party crates will link to their html_root_url if set, otherwise to the fallback location filled in by tooling, and first party will go to our own docs builds, and we avoid building any heuristics about identifying third party code into Buck itself.

@jyn514
Copy link
Member Author

jyn514 commented Mar 4, 2021

This feels like a bad change to me. A large part of the value I see in docs.rs is that it's a one-stop-location for all rustdoc output of crates.io crates.

Ok, that makes sense to me, I'll add a flag opting in to the old behavior.

For supporting Buck's use case, a better approach from rustc and rustdoc would be to accept a fallback html-root-url as part of the downstream crate's build, with lower precedence than an html_root_url attribute set by the same crate. Then we'd have reindeer and cargo-raze emit third-party targets that look like (post Buck macro expansion):

I'm not quite sure I understand - the snippet you pasted means you know at build time that serde is a third-party crate, because you pass docs.rs and not docs.local. Why is harder to tell serde is a third-party crate when building crates that depend on it? Couldn't you add some metadata field to rust_library that propagates the information? It seems weird to push that all the way back into rustdoc itself.

@jyn514 jyn514 added the T-docs-rs Relevant to the docs-rs subteam, which will review and decide on the PR/issue. label Mar 5, 2021
@jyn514
Copy link
Member Author

jyn514 commented Mar 24, 2021

ping @dtolnay - I think this may have gotten lost in your emails, I'm hoping to hear more about the Buck use case.

@dtolnay
Copy link
Member

dtolnay commented Apr 8, 2021

I'm not quite sure I understand - the snippet you pasted means you know at build time that serde is a third-party crate

Not quite. At build time (i.e. at the time that buck is executed to perform the build), it's just a rustc invocation with an opaque set of command line arguments that are not meaningful to Buck. There is no concept of "third party" baked into Buck so there is no way it would reason about what is or is not a third party crate or what the implications of the distinction are. "Third party" is only emergent from how Buck is used within some individual monorepo and how the developers have chosen to organize the code in the repo.

At the time of authoring the build targets (i.e. when the developer runs reindeer or cargo raze to translate them from upstream's Cargo.toml), definitely we know what is a third party crate, because those tools only ever operate on third party crates, so by definition "third party" == the set of targets imported via those tools. They, too, do not reason about what is or is not third party code; simply everything they operate on is regarded as third party code. The tool's job is importing third party code from the crates.io/docs.rs ecosystem into the Buck based monorepo internal ecosystem.


Why is it harder to tell serde is a third-party crate when building crates that depend on it?

Think of each Buck target (one rust_library invocation for example) as corresponding to a single invocation of a tool like rustc or rustdoc, with some arguments that possibly include output paths of the other targets it declares as dependencies, and produces one or more artifacts as outputs (like a linked executable, or an rlib, or a static lib). Sometimes the same target can be built in various "flavors" to control what the output is, for example #check to produce an rmeta only, #doc to produce rendered html documentation, and #default to produce rlib.

A small piece of Java logic in the Buck implementation of each rule (e.g. the implementation of rust_library) is responsible for rearranging the human-readable arguments from the configuration language, like name = "serde", into command line args, like '--crate-name' 'serde', or deps = ["//path/to:serde", "//path/to:serde_json"] into '--extern' 'serde=$(//path/to:serde)' '--extern' 'serde_json=$(//path/to:serde_json)', or features = ["std"] into '--cfg' 'feature="std"'. It's good when this is obvious and one-to-one, such that all the programmer is doing is writing down a straightforward rustc invocation in Buck syntax. They then get the ability to run that build in #check mode and #doc mode, build for various targets, build in ways that respect a global Buck configuration like what set of unstable features allowed, build at different optimization levels, etc.

Traversing the Buck targets of the dependencies to categorize which ones are third party is probably possible, but as you can tell is more involved than the above examples (going from features = ["std"] to '--cfg' 'feature="std"'). It involves looking at some other target's args, as opposed to its build outputs, which is suspicious.

@crlf0710 crlf0710 added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 23, 2021
@bors

This comment has been minimized.

@jyn514
Copy link
Member Author

jyn514 commented Apr 29, 2021

@dtolnay does this make things any harder for Buck? I do think this is the right change for the rest of the ecosystem, it would help with stabilizing -Z rustdoc-map and from what @SergioBenitez it sounds like rocket would also like to use it. Buck seems to have very specific requirements that I think should go through an RFC instead of me adding it as a one-off change.

@dtolnay
Copy link
Member

dtolnay commented May 11, 2021

It doesn't immediately. I raised it because the PR description discusses Buck but this isn't what Buck needs. Punting that use case seems okay.

Unrelated to Buck, it's slightly concerning that giving precedence to doc(html_root_url) leaves no way to opt out of a crate's chosen html_root_url, which means it's no longer possible for any docs.rs alternative to choose the opposite route as docs.rs on this issue, i.e. keep building everything itself and cross-linking to its own builds, as opposed to out to the internet. It's an indication that docs.rs's set of decisions are being imposed onto all other possible consumers of rustdoc, which is not necessarily wise, and an opportunity for better design that better isolates the two things. So while for my own use case this isn't a blocker, it's a new coupling that we should acknowledge (if I've understood correctly).

In particular, my sense is that docs.rs is a highly unusual consumer of rustdoc, and most other consumers of rustdoc do not have needs and priorities that are aligned with docs.rs.

  • For example docs.rs aims to support "everything" that's published to crates.io, including things with transitive dependency on bizarre requirements of the environment they're built in. This is a highly unusual requirement that only docs.rs has. In a corporate environment (Buck-based monorepo or not) we import only a tiny fraction of crates.io, and know we can build all of it, using exactly all the rust flags we use for building production code. That means some reasons for a crate to choose its own html_root_url are less applicable to us than they are to docs.rs.

  • Docs.rs is also on public internet, so whoever is accessing docs.rs is probably also able to connect to to a crate's chosen html_root_url website also on public internet. Meanwhile that's not the case for documentation hosted on intranet (firewalled or airgapped environments).

In response to your question whether this makes things harder for Buck, currently it doesn't because so far we are aiming to link to html_root_url whenever a crate has one, but if we ever needed to not do that (a crate's documentation site goes offline? a crate's documentation site does not document the set of features we primarily use?) there is no longer a way after this PR.

@crlf0710 crlf0710 added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 5, 2021
@jyn514 jyn514 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 6, 2021
@crlf0710 crlf0710 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 25, 2021
@crlf0710 crlf0710 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 17, 2021
@crlf0710 crlf0710 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 14, 2021
@jyn514 jyn514 changed the title Give precedence to doc(html_root_url) over --extern-html-root-url Give precedence to html_root_url over --extern-html-root-url by default, but add a way to opt-in to the previous behavior Aug 19, 2021
@Nemo157
Copy link
Member

Nemo157 commented Aug 20, 2021

Looks like it should work fine for docs.rs, I assume you'll manage the sequencing so that docs.rs updates to use the flag on the nightly this comes out on 😜.

@jyn514
Copy link
Member Author

jyn514 commented Aug 20, 2021

Yup, I'll wait to merge this until I have time to make a docs.rs PR. I think it would be ok in practice for docs.rs not to pass the flag for a day or two though.

@jyn514 jyn514 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 20, 2021
@jyn514
Copy link
Member Author

jyn514 commented Aug 21, 2021

@bors r=GuillaumeGomez

@bors
Copy link
Contributor

bors commented Aug 21, 2021

📌 Commit 18f0f24 has been approved by GuillaumeGomez

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 21, 2021
@bors
Copy link
Contributor

bors commented Aug 21, 2021

⌛ Testing commit 18f0f24 with merge b1928aa...

@bors
Copy link
Contributor

bors commented Aug 21, 2021

☀️ Test successful - checks-actions
Approved by: GuillaumeGomez
Pushing b1928aa to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 21, 2021
@bors bors merged commit b1928aa into rust-lang:master Aug 21, 2021
@rustbot rustbot added this to the 1.56.0 milestone Aug 21, 2021
@jyn514 jyn514 deleted the extern-url-fallback branch August 21, 2021 21:19
@ehuss
Copy link
Contributor

ehuss commented Aug 22, 2021

Sorry, I'm a bit confused on the use case here. This broke cargo's use of --extern-html-root-url. I'm skeptical that it is desired to unconditionally do one method or another for all dependencies. For example, Cargo wants to link to std's docs locally, but that uses html_root_url, so Cargo needs to pass --extern-html-root-takes-precedence. Sorry, I didn't catch the comment you left on rust-lang/cargo#8296. I don't think it was decided anywhere that it is better for the crate author to have control over where things link, so I'm not sure who this change is for. I can add --extern-html-root-takes-precedence, but I'm a little confused. Was this specifically for Buck? I'm having trouble following the comments above.

ehuss added a commit to ehuss/cargo that referenced this pull request Aug 22, 2021
A change in rust-lang/rust#82776 broke this test,
so disabling for now until it is figured out how things are going to work.
bors added a commit to rust-lang/cargo that referenced this pull request Aug 23, 2021
Temporarily disable extern-html-root-url test.

A change in rust-lang/rust#82776 broke this test,
so disabling for now until it is figured out how things are going to work.
ehuss pushed a commit to ehuss/cargo that referenced this pull request Aug 23, 2021
Temporarily disable extern-html-root-url test.

A change in rust-lang/rust#82776 broke this test,
so disabling for now until it is figured out how things are going to work.
jyn514 added a commit to jyn514/rust that referenced this pull request Sep 4, 2021
…=GuillaumeGomez"

This reverts commit b1928aa, reversing
changes made to 99b73e8.
@jyn514
Copy link
Member Author

jyn514 commented Sep 4, 2021

This broke cargo's use of --extern-html-root-url. I'm skeptical that it is desired to unconditionally do one method or another for all dependencies. For example, Cargo wants to link to std's docs locally, but that uses html_root_url, so Cargo needs to pass --extern-html-root-takes-precedence.

I thought you said in rust-lang/cargo#8296 that cargo needed a way to give html_root_url precedence before stabilizing -Zrustdoc-map :/

Should there be some way for the crate author to control the link destination? The documentation URL set in Cargo.toml is never consulted. I think this is unlikely to work with rustdoc, because it expects a certain structure to the API docs, and documentation URLs often point to user guides, or other things that aren't API docs. The #![doc(html_root_url = "...")] attribute is overridden by the --extern-html-root-url flag. This may be a good or bad thing depending on your perspective (should the user have control, or should the crate author?).

Is there something else that needs to happen? My long term goal is to stabilize -Zrustdoc-map and turn it on by default so that cargo doc --no-deps is no longer as broken.

@ehuss
Copy link
Contributor

ehuss commented Sep 4, 2021

Ah, apologies for the confusion. I think in rust-lang/cargo#8296 I was mainly asking a question of what the correct behavior should be. I don't know the answer. I think there are legitimate arguments for both approaches (although this was good to highlight that unconditionally making the attribute take precedence prevents serving std docs locally).

I'll try to think it over and see if there is some kind of compromise. The flag here (--extern-html-root-takes-precedence) might be fine, I was just trying to understand what this was being added for. If it was for cargo, then I think this PR wasn't what we really needed. But if it was added for other users, then that would be good to know those use cases.

@jyn514
Copy link
Member Author

jyn514 commented Sep 4, 2021

I think it helped Rocket the most: rust-lang/api-guidelines#230 (comment). I would also be ok with just changing the default back but leaving a way for you to opt-in to the other precedence; but if that only helps Rocket it might not be worth the complexity.

wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request Oct 22, 2021
Pkgsrc changes:
 * Remove one now-longer-applicable patch, adjust a few others
 * Bump bootstrap requirements to 1.55.0.

Upstream changes:

Version 1.56.0 (2021-10-21)
========================

Language
--------

- [The 2021 Edition is now stable.][rust#88100]
  See [the edition guide][rust-2021-edition-guide] for more details.
- [The pattern in `binding @ pattern` can now also introduce new bindings.]
  [rust#85305]
- [Union field access is permitted in `const fn`.][rust#85769]

[rust-2021-edition-guide]: https://doc.rust-lang.org/nightly/edition-guide/rust-2021/index.html

Compiler
--------

- [Upgrade to LLVM 13.][rust#87570]
- [Support memory, address, and thread sanitizers on
  aarch64-unknown-freebsd.][rust#88023]
- [Allow specifying a deployment target version for all iOS targets][rust#87699]
- [Warnings can be forced on with `--force-warn`.][rust#87472]
  This feature is primarily intended for usage by `cargo fix`,
  rather than end users.
- [Promote `aarch64-apple-ios-sim` to Tier 2\*.][rust#87760]
- [Add `powerpc-unknown-freebsd` at Tier 3\*.][rust#87370]
- [Add `riscv32imc-esp-espidf` at Tier 3\*.][rust#87666]

\* Refer to Rust's [platform support page][platform-support-doc] for more
information on Rust's tiered platform support.

Libraries
---------

- [Allow writing of incomplete UTF-8 sequences via stdout/stderr on Windows.]
  [rust#83342]
  The Windows console still requires valid Unicode, but this change allows
  splitting a UTF-8 character across multiple write calls. This allows, for
  instance, programs that just read and write data buffers (e.g. copying a file
  to stdout) without regard for Unicode or character boundaries.
- [Prefer `AtomicU{64,128}` over Mutex for Instant backsliding protection.]
  [rust#83093]
  For this use case, atomics scale much better under contention.
- [Implement `Extend<(A, B)>` for `(Extend<A>, Extend<B>)`][rust#85835]
- [impl Default, Copy, Clone for std::io::Sink and std::io::Empty][rust#86744]
- [`impl From<[(K, V); N]>` for all collections.][rust#84111]
- [Remove `P: Unpin` bound on impl Future for Pin.][rust#81363]
- [Treat invalid environment variable names as non-existent.][rust#86183]
  Previously, the environment functions would panic if given a
  variable name with an internal null character or equal sign (`=`).
  Now, these functions will just treat such names as non-existent
  variables, since the OS cannot represent the existence of a
  variable with such a name.

Stabilised APIs
---------------

- [`std::os::unix::fs::chroot`]
- [`UnsafeCell::raw_get`]
- [`BufWriter::into_parts`]
- [`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]
  These APIs were previously stable in `std`, but are now also available
  in `core`.
- [`Vec::shrink_to`]
- [`String::shrink_to`]
- [`OsString::shrink_to`]
- [`PathBuf::shrink_to`]
- [`BinaryHeap::shrink_to`]
- [`VecDeque::shrink_to`]
- [`HashMap::shrink_to`]
- [`HashSet::shrink_to`]

These APIs are now usable in const contexts:

- [`std::mem::transmute`]
- [`[T]::first`][`slice::first`]
- [`[T]::split_first`][`slice::split_first`]
- [`[T]::last`][`slice::last`]
- [`[T]::split_last`][`slice::split_last`]

Cargo
-----

- [Cargo supports specifying a minimum supported Rust version in Cargo.toml.]
  [`rust-version`] This has no effect at present on dependency
  version selection.  We encourage crates to specify their minimum
  supported Rust version, and we encourage CI systems that support
  Rust code to include a crate's specified minimum version in the
  text matrix for that crate by default.

Compatibility notes
-------------------

- [Update to new argument parsing rules on Windows.][rust#87580]
  This adjusts Rust's standard library to match the behavior of the standard
  libraries for C/C++. The rules have changed slightly over time, and this PR
  brings us to the latest set of rules (changed in 2008).
- [Disallow the aapcs calling convention on aarch64][rust#88399]
  This was already not supported by LLVM; this change surfaces this lack of
  support with a better error message.
- [Make `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` warn by default][rust#87385]
- [Warn when an escaped newline skips multiple lines.][rust#87671]
- [Calls to `libc::getpid` / `std::process::id` from `Command::pre_exec`
  may return different values on glibc <= 2.24.][rust#81825] Rust
  now invokes the `clone3` system call directly, when available,
  to use new functionality available via that system call. Older
  versions of glibc cache the result of `getpid`, and only update
  that cache when calling glibc's clone/fork functions, so a direct
  system call bypasses that cache update. glibc 2.25 and newer no
  longer cache `getpid` for exactly this reason.

Internal changes
----------------
These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

- [LLVM is compiled with PGO in published x86_64-unknown-linux-gnu
  artifacts.][rust#88069] This improves the performance of most Rust builds.

- [Unify representation of macros in internal data structures.][rust#88019]
  This change fixes a host of bugs with the handling of macros by the compiler,
  as well as rustdoc.

[`std::os::unix::fs::chroot`]: https://doc.rust-lang.org/stable/std/os/unix/fs/fn.chroot.html
[`Iterator::intersperse`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`Iterator::intersperse_with`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`UnsafeCell::raw_get`]: https://doc.rust-lang.org/stable/std/cell/struct.UnsafeCell.html#method.raw_get
[`BufWriter::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.BufWriter.html#method.into_parts
[`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]: rust-lang/rust#84662
[`Vec::shrink_to`]: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.shrink_to
[`String::shrink_to`]: https://doc.rust-lang.org/stable/std/string/struct.String.html#method.shrink_to
[`OsString::shrink_to`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.shrink_to
[`PathBuf::shrink_to`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.shrink_to
[`BinaryHeap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.shrink_to
[`VecDeque::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.VecDeque.html#method.shrink_to
[`HashMap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_map/struct.HashMap.html#method.shrink_to
[`HashSet::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_set/struct.HashSet.html#method.shrink_to
[`std::mem::transmute`]: https://doc.rust-lang.org/stable/std/mem/fn.transmute.html
[`slice::first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.first
[`slice::split_first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_first
[`slice::last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.last
[`slice::split_last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_last
[`rust-version`]: https://doc.rust-lang.org/nightly/cargo/reference/manifest.html#the-rust-version-field
[rust#87671]: rust-lang/rust#87671
[rust#86183]: rust-lang/rust#86183
[rust#87385]: rust-lang/rust#87385
[rust#88100]: rust-lang/rust#88100
[rust#86860]: rust-lang/rust#86860
[rust#84039]: rust-lang/rust#84039
[rust#86492]: rust-lang/rust#86492
[rust#88363]: rust-lang/rust#88363
[rust#85305]: rust-lang/rust#85305
[rust#87832]: rust-lang/rust#87832
[rust#88069]: rust-lang/rust#88069
[rust#87472]: rust-lang/rust#87472
[rust#87699]: rust-lang/rust#87699
[rust#87570]: rust-lang/rust#87570
[rust#88023]: rust-lang/rust#88023
[rust#87760]: rust-lang/rust#87760
[rust#87370]: rust-lang/rust#87370
[rust#87580]: rust-lang/rust#87580
[rust#83342]: rust-lang/rust#83342
[rust#83093]: rust-lang/rust#83093
[rust#88177]: rust-lang/rust#88177
[rust#88548]: rust-lang/rust#88548
[rust#88551]: rust-lang/rust#88551
[rust#88299]: rust-lang/rust#88299
[rust#88220]: rust-lang/rust#88220
[rust#85835]: rust-lang/rust#85835
[rust#86879]: rust-lang/rust#86879
[rust#86744]: rust-lang/rust#86744
[rust#84662]: rust-lang/rust#84662
[rust#86593]: rust-lang/rust#86593
[rust#81050]: rust-lang/rust#81050
[rust#81363]: rust-lang/rust#81363
[rust#84111]: rust-lang/rust#84111
[rust#85769]: rust-lang/rust#85769 (comment)
[rust#88490]: rust-lang/rust#88490
[rust#88269]: rust-lang/rust#88269
[rust#84176]: rust-lang/rust#84176
[rust#88399]: rust-lang/rust#88399
[rust#88227]: rust-lang/rust#88227
[rust#88200]: rust-lang/rust#88200
[rust#82776]: rust-lang/rust#82776
[rust#88077]: rust-lang/rust#88077
[rust#87728]: rust-lang/rust#87728
[rust#87050]: rust-lang/rust#87050
[rust#87619]: rust-lang/rust#87619
[rust#81825]: rust-lang/rust#81825 (comment)
[rust#88019]: rust-lang/rust#88019
[rust#87666]: rust-lang/rust#87666
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Nov 20, 2021
Pkgsrc changes:
 * Bump bootstrap kit version to 1.55.0.
 * Adjust patches as needed, some no longer apply (so removed)
 * Update checksum adjustments.
 * Avoid rust-llvm on SunOS
 * Optionally build docs
 * Remove reference to closed/old PR#54621

Upstream changes:

Version 1.56.1 (2021-11-01)
===========================

- New lints to detect the presence of bidirectional-override Unicode
  codepoints in the compiled source code ([CVE-2021-42574])

[CVE-2021-42574]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-42574

Version 1.56.0 (2021-10-21)
========================

Language
--------

- [The 2021 Edition is now stable.][rust#88100]
  See [the edition guide][rust-2021-edition-guide] for more details.
- [The pattern in `binding @ pattern` can now also introduce new bindings.]
  [rust#85305]
- [Union field access is permitted in `const fn`.][rust#85769]

[rust-2021-edition-guide]:
  https://doc.rust-lang.org/nightly/edition-guide/rust-2021/index.html

Compiler
--------

- [Upgrade to LLVM 13.][rust#87570]
- [Support memory, address, and thread sanitizers on aarch64-unknown-freebsd.]
  [rust#88023]
- [Allow specifying a deployment target version for all iOS targets][rust#87699]
- [Warnings can be forced on with `--force-warn`.][rust#87472]
  This feature is primarily intended for usage by `cargo fix`, rather than
  end users.
- [Promote `aarch64-apple-ios-sim` to Tier 2\*.][rust#87760]
- [Add `powerpc-unknown-freebsd` at Tier 3\*.][rust#87370]
- [Add `riscv32imc-esp-espidf` at Tier 3\*.][rust#87666]

\* Refer to Rust's [platform support page][platform-support-doc] for more
information on Rust's tiered platform support.

Libraries
---------

- [Allow writing of incomplete UTF-8 sequences via stdout/stderr on Windows.]
  [rust#83342]
  The Windows console still requires valid Unicode, but this change allows
  splitting a UTF-8 character across multiple write calls. This allows, for
  instance, programs that just read and write data buffers (e.g. copying a file
  to stdout) without regard for Unicode or character boundaries.
- [Prefer `AtomicU{64,128}` over Mutex for Instant backsliding protection.]
  [rust#83093]
  For this use case, atomics scale much better under contention.
- [Implement `Extend<(A, B)>` for `(Extend<A>, Extend<B>)`][rust#85835]
- [impl Default, Copy, Clone for std::io::Sink and std::io::Empty][rust#86744]
- [`impl From<[(K, V); N]>` for all collections.][rust#84111]
- [Remove `P: Unpin` bound on impl Future for Pin.][rust#81363]
- [Treat invalid environment variable names as non-existent.][rust#86183]
  Previously, the environment functions would panic if given a
  variable name with an internal null character or equal sign (`=`).
  Now, these functions will just treat such names as non-existent
  variables, since the OS cannot represent the existence of a
  variable with such a name.

Stabilised APIs
---------------

- [`std::os::unix::fs::chroot`]
- [`UnsafeCell::raw_get`]
- [`BufWriter::into_parts`]
- [`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]
  These APIs were previously stable in `std`, but are now also available
  in `core`.
- [`Vec::shrink_to`]
- [`String::shrink_to`]
- [`OsString::shrink_to`]
- [`PathBuf::shrink_to`]
- [`BinaryHeap::shrink_to`]
- [`VecDeque::shrink_to`]
- [`HashMap::shrink_to`]
- [`HashSet::shrink_to`]

These APIs are now usable in const contexts:

- [`std::mem::transmute`]
- [`[T]::first`][`slice::first`]
- [`[T]::split_first`][`slice::split_first`]
- [`[T]::last`][`slice::last`]
- [`[T]::split_last`][`slice::split_last`]

Cargo
-----

- [Cargo supports specifying a minimum supported Rust version in Cargo.toml.]
  [`rust-version`]
  This has no effect at present on dependency version selection.
  We encourage crates to specify their minimum supported Rust
  version, and we encourage CI systems that support Rust code to
  include a crate's specified minimum version in the text matrix
  for that crate by default.

Compatibility notes
-------------------

- [Update to new argument parsing rules on Windows.][rust#87580]
  This adjusts Rust's standard library to match the behavior of the standard
  libraries for C/C++. The rules have changed slightly over time, and this PR
  brings us to the latest set of rules (changed in 2008).
- [Disallow the aapcs calling convention on aarch64][rust#88399]
  This was already not supported by LLVM; this change surfaces this lack of
  support with a better error message.
- [Make `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` warn by default][rust#87385]
- [Warn when an escaped newline skips multiple lines.][rust#87671]
- [Calls to `libc::getpid` / `std::process::id` from `Command::pre_exec`
  may return different values on glibc <= 2.24.][rust#81825]
  Rust now invokes the `clone3` system call directly, when available,
  to use new functionality available via that system call. Older
  versions of glibc cache the result of `getpid`, and only update
  that cache when calling glibc's clone/fork functions, so a direct
  system call bypasses that cache update. glibc 2.25 and newer no
  longer cache `getpid` for exactly this reason.

Internal changes
----------------
These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

- [LLVM is compiled with PGO in published x86_64-unknown-linux-gnu artifacts.]
  [rust#88069]
  This improves the performance of most Rust builds.
- [Unify representation of macros in internal data structures.][rust#88019]
  This change fixes a host of bugs with the handling of macros by the compiler,
  as well as rustdoc.

[`std::os::unix::fs::chroot`]: https://doc.rust-lang.org/stable/std/os/unix/fs/fn.chroot.html
[`Iterator::intersperse`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`Iterator::intersperse_with`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`UnsafeCell::raw_get`]: https://doc.rust-lang.org/stable/std/cell/struct.UnsafeCell.html#method.raw_get
[`BufWriter::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.BufWriter.html#method.into_parts
[`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]: rust-lang/rust#84662
[`Vec::shrink_to`]: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.shrink_to
[`String::shrink_to`]: https://doc.rust-lang.org/stable/std/string/struct.String.html#method.shrink_to
[`OsString::shrink_to`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.shrink_to
[`PathBuf::shrink_to`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.shrink_to
[`BinaryHeap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.shrink_to
[`VecDeque::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.VecDeque.html#method.shrink_to
[`HashMap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_map/struct.HashMap.html#method.shrink_to
[`HashSet::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_set/struct.HashSet.html#method.shrink_to
[`std::mem::transmute`]: https://doc.rust-lang.org/stable/std/mem/fn.transmute.html
[`slice::first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.first
[`slice::split_first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_first
[`slice::last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.last
[`slice::split_last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_last
[`rust-version`]: https://doc.rust-lang.org/nightly/cargo/reference/manifest.html#the-rust-version-field
[rust#87671]: rust-lang/rust#87671
[rust#86183]: rust-lang/rust#86183
[rust#87385]: rust-lang/rust#87385
[rust#88100]: rust-lang/rust#88100
[rust#86860]: rust-lang/rust#86860
[rust#84039]: rust-lang/rust#84039
[rust#86492]: rust-lang/rust#86492
[rust#88363]: rust-lang/rust#88363
[rust#85305]: rust-lang/rust#85305
[rust#87832]: rust-lang/rust#87832
[rust#88069]: rust-lang/rust#88069
[rust#87472]: rust-lang/rust#87472
[rust#87699]: rust-lang/rust#87699
[rust#87570]: rust-lang/rust#87570
[rust#88023]: rust-lang/rust#88023
[rust#87760]: rust-lang/rust#87760
[rust#87370]: rust-lang/rust#87370
[rust#87580]: rust-lang/rust#87580
[rust#83342]: rust-lang/rust#83342
[rust#83093]: rust-lang/rust#83093
[rust#88177]: rust-lang/rust#88177
[rust#88548]: rust-lang/rust#88548
[rust#88551]: rust-lang/rust#88551
[rust#88299]: rust-lang/rust#88299
[rust#88220]: rust-lang/rust#88220
[rust#85835]: rust-lang/rust#85835
[rust#86879]: rust-lang/rust#86879
[rust#86744]: rust-lang/rust#86744
[rust#84662]: rust-lang/rust#84662
[rust#86593]: rust-lang/rust#86593
[rust#81050]: rust-lang/rust#81050
[rust#81363]: rust-lang/rust#81363
[rust#84111]: rust-lang/rust#84111
[rust#85769]: rust-lang/rust#85769 (comment)
[rust#88490]: rust-lang/rust#88490
[rust#88269]: rust-lang/rust#88269
[rust#84176]: rust-lang/rust#84176
[rust#88399]: rust-lang/rust#88399
[rust#88227]: rust-lang/rust#88227
[rust#88200]: rust-lang/rust#88200
[rust#82776]: rust-lang/rust#82776
[rust#88077]: rust-lang/rust#88077
[rust#87728]: rust-lang/rust#87728
[rust#87050]: rust-lang/rust#87050
[rust#87619]: rust-lang/rust#87619
[rust#81825]: rust-lang/rust#81825 (comment)
[rust#88019]: rust-lang/rust#88019
[rust#87666]: rust-lang/rust#87666

Version 1.55.0 (2021-09-09)
============================

Language
--------
- [You can now write open "from" range patterns (`X..`), which will start
  at `X` and will end at the maximum value of the integer.][83918]
- [You can now explicitly import the prelude of different editions
  through `std::prelude` (e.g. `use std::prelude::rust_2021::*;`).][86294]

Compiler
--------
- [Added tier 3\* support for `powerpc64le-unknown-freebsd`.][83572]

\* Refer to Rust's [platform support page][platform-support-doc] for more
   information on Rust's tiered platform support.

Libraries
---------

- [Updated std's float parsing to use the Eisel-Lemire algorithm.][86761]
  These improvements should in general provide faster string parsing of floats,
  no longer reject certain valid floating point values, and reduce
  the produced code size for non-stripped artifacts.
- [`string::Drain` now implements `AsRef<str>` and `AsRef<[u8]>`.][86858]

Stabilised APIs
---------------

- [`Bound::cloned`]
- [`Drain::as_str`]
- [`IntoInnerError::into_error`]
- [`IntoInnerError::into_parts`]
- [`MaybeUninit::assume_init_mut`]
- [`MaybeUninit::assume_init_ref`]
- [`MaybeUninit::write`]
- [`array::map`]
- [`ops::ControlFlow`]
- [`x86::_bittest`]
- [`x86::_bittestandcomplement`]
- [`x86::_bittestandreset`]
- [`x86::_bittestandset`]
- [`x86_64::_bittest64`]
- [`x86_64::_bittestandcomplement64`]
- [`x86_64::_bittestandreset64`]
- [`x86_64::_bittestandset64`]

The following previously stable functions are now `const`.

- [`str::from_utf8_unchecked`]


Cargo
-----
- [Cargo will now deduplicate compiler diagnostics to the terminal when invoking
  rustc in parallel such as when using `cargo test`.][cargo/9675]
- [The package definition in `cargo metadata` now includes the `"default_run"`
  field from the manifest.][cargo/9550]
- [Added `cargo d` as an alias for `cargo doc`.][cargo/9680]
- [Added `{lib}` as formatting option for `cargo tree` to print the `"lib_name"`
  of packages.][cargo/9663]

Rustdoc
-------
- [Added "Go to item on exact match" search option.][85876]
- [The "Implementors" section on traits no longer shows redundant
  method definitions.][85970]
- [Trait implementations are toggled open by default.][86260] This should
  make the implementations more searchable by tools like `CTRL+F` in
  your browser.
- [Intra-doc links should now correctly resolve associated items (e.g. methods)
  through type aliases.][86334]
- [Traits which are marked with `#[doc(hidden)]` will no longer appear in the
  "Trait Implementations" section.][86513]


Compatibility Notes
-------------------
- [std functions that return an `io::Error` will no longer use the
  `ErrorKind::Other` variant.][85746] This is to better reflect that these
  kinds of errors could be categorised [into newer more specific `ErrorKind`
  variants][79965], and that they do not represent a user error.
- [Using environment variable names with `process::Command` on Windows now
  behaves as expected.][85270] Previously using envionment variables with
  `Command` would cause them to be ASCII-uppercased.
- [Rustdoc will now warn on using rustdoc lints that aren't prefixed
  with `rustdoc::`][86849]

[86849]: rust-lang/rust#86849
[86513]: rust-lang/rust#86513
[86334]: rust-lang/rust#86334
[86260]: rust-lang/rust#86260
[85970]: rust-lang/rust#85970
[85876]: rust-lang/rust#85876
[83572]: rust-lang/rust#83572
[86294]: rust-lang/rust#86294
[86858]: rust-lang/rust#86858
[86761]: rust-lang/rust#86761
[85769]: rust-lang/rust#85769
[85746]: rust-lang/rust#85746
[85305]: rust-lang/rust#85305
[85270]: rust-lang/rust#85270
[84111]: rust-lang/rust#84111
[83918]: rust-lang/rust#83918
[79965]: rust-lang/rust#79965
[87370]: rust-lang/rust#87370
[87298]: rust-lang/rust#87298
[cargo/9663]: rust-lang/cargo#9663
[cargo/9675]: rust-lang/cargo#9675
[cargo/9550]: rust-lang/cargo#9550
[cargo/9680]: rust-lang/cargo#9680
[cargo/9663]: rust-lang/cargo#9663
[`array::map`]: https://doc.rust-lang.org/stable/std/primitive.array.html#method.map
[`Bound::cloned`]: https://doc.rust-lang.org/stable/std/ops/enum.Bound.html#method.cloned
[`Drain::as_str`]: https://doc.rust-lang.org/stable/std/string/struct.Drain.html#method.as_str
[`IntoInnerError::into_error`]: https://doc.rust-lang.org/stable/std/io/struct.IntoInnerError.html#method.into_error
[`IntoInnerError::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.IntoInnerError.html#method.into_parts
[`MaybeUninit::assume_init_mut`]: https://doc.rust-lang.org/stable/std/mem/union.MaybeUninit.html#method.assume_init_mut
[`MaybeUninit::assume_init_ref`]: https://doc.rust-lang.org/stable/std/mem/union.MaybeUninit.html#method.assume_init_ref
[`MaybeUninit::write`]: https://doc.rust-lang.org/stable/std/mem/union.MaybeUninit.html#method.write
[`Seek::rewind`]: https://doc.rust-lang.org/stable/std/io/trait.Seek.html#method.rewind
[`ops::ControlFlow`]: https://doc.rust-lang.org/stable/std/ops/enum.ControlFlow.html
[`str::from_utf8_unchecked`]: https://doc.rust-lang.org/stable/std/str/fn.from_utf8_unchecked.html
[`x86::_bittest`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittest.html
[`x86::_bittestandcomplement`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittestandcomplement.html
[`x86::_bittestandreset`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittestandreset.html
[`x86::_bittestandset`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittestandset.html
[`x86_64::_bittest64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittest64.html
[`x86_64::_bittestandcomplement64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittestandcomplement64.html
[`x86_64::_bittestandreset64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittestandreset64.html
[`x86_64::_bittestandset64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittestandset64.html
@dtolnay dtolnay assigned dtolnay and unassigned GuillaumeGomez Mar 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. requires-nightly This issue requires a nightly compiler in some way. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-docs-rs Relevant to the docs-rs subteam, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants