-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add split-debuginfo profile option #9112
Conversation
r? @Eh2406 (rust-highfive has picked a reviewer for you, use r? to override) |
This will fail CI while I'm still figuring out tests (and waiting on a nightly with support for this option), but I wanted to go ahead and open this earlier to see what others' thoughts on this were. I think that we really do need to change the defaults here despite the possibly breaking change, they can have a massive effect on local development. For comparison Cargo's binary builds in 10.5s with dsymutil enabled (packed debuginfo, historical default), and 2.8s without (and most of that time is spent in the linker). I also think that if we do this right for Linux builds with split-dwarf we'll be able to drive down the size of target directories with less duplicated debuginfo as well. |
r? @ehuss |
I am a bit surprised to see this as a profile option rather than something you configure in e.g. .cargo/config - it seems like the kind of thing I would want to set on a system/user level rather than per-project. Am I wrong about that? I suppose making it a profile option is maximally flexible though, so in that case seems fine, just want to better understand the use case here - e.g., would we expect someone to publish to crates.io with this set? |
Profile configuration can happen in a number of places nowadays, so by making it a profile option you can configure it via |
This commit adds a new `split-debuginfo` option to Cargo compilation profiles which gets forwarded to the `-Csplit-debuginfo` codegen option in rustc. This commit also sets the default, only on macOS, to be `-Csplit-debuginfo=unpacked`. The purpose of this change is to leverage rust-lang/rust#79570 to avoid running `dsymutil` on incremental builds while also preserving a pleasant debugging experience by default. This should lead to much faster incremental build times on macOS since `dsymutil` isn't exactly the speediest tool in the world. This is technically a breaking change in Cargo because we're no longer by-default producing the `*.dSYM` folders on macOS. If those are still desired, however, authors can always run `dsymutil` themselves or otherwise configure `split-debuginfo = 'packed'` in their manifest/profile configuration.
5e56def
to
dd58144
Compare
This commit updates the rustc info cache to cache failures to execute rustc as well as successes. This fixes a weird issue where if you're probing for flags the `rustc_info_cache` test fails on channels which don't have the flag since previously a failure to execute rustc resulted in never caching the result.
dd58144
to
cc5e9df
Compare
Ok this now has all green tests. The main thing I did was to update the rustc caching to cache failure statuses as well as successful statuses when we learn about rustc args so we can cache the failure to use |
1e19f08
to
59d59af
Compare
We discussed this at the Cargo meeting today and the conclusion was that we'll:
|
src/cargo/ops/cargo_compile.rs
Outdated
let profile = profiles.get_profile( | ||
pkg.package_id(), | ||
ws.is_member(pkg), | ||
is_local, | ||
unit_for, | ||
target_mode, | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not clear to me why this moved inside the loop.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh this was a relic of when this was doing a macOS-specific thing internally so the CompileKind
was passed in, I undid that though so I just undid this as well.
let profile = profiles.get_profile( | ||
pkg.package_id(), | ||
/*is_member*/ false, | ||
/*is_local*/ false, | ||
unit_for, | ||
mode, | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same question about why this moved.
59d59af
to
ffa9dbd
Compare
🎉 🎉 🎉 🎉 🎉 @bors r+ |
📌 Commit ffa9dbd has been approved by |
☀️ Test successful - checks-actions |
Update cargo 5 commits in e099df243bb2495b9b197f79c19f124032b1e778..34170fcd6e0947808a1ac63ac85ffc0da7dace2f 2021-02-01 16:24:34 +0000 to 2021-02-04 15:52:52 +0000 - Fix permission issue with `cargo vendor`. (rust-lang/cargo#9131) - Add split-debuginfo profile option (rust-lang/cargo#9112) - Add RegistryBuilder for tests, and update crates-io error handling. (rust-lang/cargo#9126) - Add some documentation for index and registry stuff. (rust-lang/cargo#9125) - Fix env/cfg set for `cargo test` and `cargo run`. (rust-lang/cargo#9122)
I've posted about this on internals for the next step hopefully in turning this on by default. |
This option is a string and acceptable values are the same as those the | ||
[compiler accepts][`-C split-debuginfo` flag]. This option is currently not | ||
passed by default on platforms other than macOS, where it defaults to | ||
`unpacked`. The default may change in the future for platforms using DWARF |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ehuss looks like this wasn't changed with the rest of the PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Opened #9218 about this.
This commit continues the work from rust-lang#9112 to enable `unpacked` split debuginfo on macOS targets by default. This has been discussed on [internals] for awhile now and no breakage has emerged while significant speedups have. This is expected to be a compile-time and `target`-directory size win for almost all macOS Rust projects. While breakage is possible it's possible to mitigate this with project-local or global cargo configuration of the `dev` and `test` profiles. [internals]: https://internals.rust-lang.org/t/help-test-faster-incremental-debug-macos-builds-on-nightly/14016/9
Default macOS targets to `unpacked` debuginfo This commit continues the work from #9112 to enable `unpacked` split debuginfo on macOS targets by default. This has been discussed on [internals] for awhile now and no breakage has emerged while significant speedups have. This is expected to be a compile-time and `target`-directory size win for almost all macOS Rust projects. While breakage is possible it's possible to mitigate this with project-local or global cargo configuration of the `dev` and `test` profiles. [internals]: https://internals.rust-lang.org/t/help-test-faster-incremental-debug-macos-builds-on-nightly/14016/9
Pkgsrc changes: * Remove one SunOS patch, apparently no longer needed. * Adapt one patch for Darwin, adjust cargo checksum accordingly. * Adjust bootstraps to version 1.50.0. Version 1.51.0 (2021-03-25) ============================ Language -------- - [You can now parameterize items such as functions, traits, and `struct`s by constant values in addition to by types and lifetimes.][79135] Also known as "const generics" E.g. you can now write the following. Note: Only values of primitive integers, `bool`, or `char` types are currently permitted. ```rust struct GenericArray<T, const LENGTH: usize> { inner: [T; LENGTH] } impl<T, const LENGTH: usize> GenericArray<T, LENGTH> { const fn last(&self) -> Option<&T> { if LENGTH == 0 { None } else { Some(&self.inner[LENGTH - 1]) } } } ``` Compiler -------- - [Added the `-Csplit-debuginfo` codegen option for macOS platforms.][79570] This option controls whether debug information is split across multiple files or packed into a single file. **Note** This option is unstable on other platforms. - [Added tier 3\* support for `aarch64_be-unknown-linux-gnu`, `aarch64-unknown-linux-gnu_ilp32`, and `aarch64_be-unknown-linux-gnu_ilp32` targets.][81455] - [Added tier 3 support for `i386-unknown-linux-gnu` and `i486-unknown-linux-gnu` targets.][80662] - [The `target-cpu=native` option will now detect individual features of CPUs.][80749] - [Rust now uses `inline-asm` for stack probes when used with LLVM 11.0.1+][77885] \* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support. Libraries --------- - [`Box::downcast` is now also implemented for any `dyn Any + Send + Sync` object.][80945] - [`str` now implements `AsMut<str>`.][80279] - [`u64` and `u128` now implement `From<char>`.][79502] - [`Error` is now implemented for `&T` where `T` implements `Error`.][75180] - [`Poll::{map_ok, map_err}` are now implemented for `Poll<Option<Result<T, E>>>`.][80968] - [`unsigned_abs` is now implemented for all signed integer types.][80959] - [`io::Empty` now implements `io::Seek`.][78044] - [`rc::Weak<T>` and `sync::Weak<T>`'s methods such as `as_ptr` are now implemented for `T: ?Sized` types.][80764] Stabilized APIs --------------- - [`Arc::decrement_strong_count`] - [`Arc::increment_strong_count`] - [`Once::call_once_force`] - [`Peekable::next_if_eq`] - [`Peekable::next_if`] - [`Seek::stream_position`] - [`array::IntoIter`] - [`panic::panic_any`] - [`ptr::addr_of!`] - [`ptr::addr_of_mut!`] - [`slice::fill_with`] - [`slice::split_inclusive_mut`] - [`slice::split_inclusive`] - [`slice::strip_prefix`] - [`slice::strip_suffix`] - [`str::split_inclusive`] - [`sync::OnceState`] - [`task::Wake`] Cargo ----- - [Added the `split-debuginfo` profile option to control the -Csplit-debuginfo codegen option.][cargo/9112] - [Added the `resolver` field to `Cargo.toml` to enable the new feature resolver and CLI option behavior.][cargo/8997] Version 2 of the feature resolver will try to avoid unifying features of dependencies where that unification could be unwanted. Such as using the same dependency with a `std` feature in a build scripts and proc-macros, while using the `no-std` feature in the final binary. See the [Cargo book documentation][feature-resolver@2.0] for more information on the feature. Rustdoc ------- - [Rustdoc will now include documentation for methods available from `Deref` traits.][80653] - [You can now provide a `--default-theme` flag which sets the default theme to use for documentation.][79642] Various improvements to intra-doc links: - [You can link to non-path primitives such as `slice`.][80181] - [You can link to associated items.][74489] - [You can now include generic parameters when linking to items, like `Vec<T>`.][76934] Misc ---- - [You can now pass `--include-ignored` to tests (e.g. with `cargo test -- --include-ignored`) to include testing tests marked `#[ignore]`.][80053] Compatibility Notes ------------------- - [WASI platforms no longer use the `wasm-bindgen` ABI, and instead use the wasm32 ABI.][79998] - [`rustc` no longer promotes division, modulo and indexing operations to `const` that could fail.][80579] - [The minimum version of glibc for the following platforms has been bumped to version 2.31 for the distributed artifacts.][81521] - `armv5te-unknown-linux-gnueabi` - `sparc64-unknown-linux-gnu` - `thumbv7neon-unknown-linux-gnueabihf` - `armv7-unknown-linux-gnueabi` - `x86_64-unknown-linux-gnux32` Internal Only ------------- - [Consistently avoid constructing optimized MIR when not doing codegen][80718] [79135]: rust-lang/rust#79135 [74489]: rust-lang/rust#74489 [76934]: rust-lang/rust#76934 [79570]: rust-lang/rust#79570 [80181]: rust-lang/rust#80181 [79642]: rust-lang/rust#79642 [80945]: rust-lang/rust#80945 [80279]: rust-lang/rust#80279 [80053]: rust-lang/rust#80053 [79502]: rust-lang/rust#79502 [75180]: rust-lang/rust#75180 [79135]: rust-lang/rust#79135 [81521]: rust-lang/rust#81521 [80968]: rust-lang/rust#80968 [80959]: rust-lang/rust#80959 [80718]: rust-lang/rust#80718 [80653]: rust-lang/rust#80653 [80579]: rust-lang/rust#80579 [79998]: rust-lang/rust#79998 [78044]: rust-lang/rust#78044 [81455]: rust-lang/rust#81455 [80764]: rust-lang/rust#80764 [80749]: rust-lang/rust#80749 [80662]: rust-lang/rust#80662 [77885]: rust-lang/rust#77885 [cargo/8997]: rust-lang/cargo#8997 [cargo/9112]: rust-lang/cargo#9112 [feature-resolver@2.0]: https://doc.rust-lang.org/nightly/cargo/reference/features.html#feature-resolver-version-2 [`Once::call_once_force`]: https://doc.rust-lang.org/stable/std/sync/struct.Once.html#method.call_once_force [`sync::OnceState`]: https://doc.rust-lang.org/stable/std/sync/struct.OnceState.html [`panic::panic_any`]: https://doc.rust-lang.org/stable/std/panic/fn.panic_any.html [`slice::strip_prefix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`slice::strip_suffix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`Arc::increment_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.increment_strong_count [`Arc::decrement_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.decrement_strong_count [`slice::fill_with`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.fill_with [`ptr::addr_of!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of.html [`ptr::addr_of_mut!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of_mut.html [`array::IntoIter`]: https://doc.rust-lang.org/nightly/std/array/struct.IntoIter.html [`slice::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive [`slice::split_inclusive_mut`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive_mut [`str::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.str.html#method.split_inclusive [`task::Wake`]: https://doc.rust-lang.org/nightly/std/task/trait.Wake.html [`Seek::stream_position`]: https://doc.rust-lang.org/nightly/std/io/trait.Seek.html#method.stream_position [`Peekable::next_if`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if [`Peekable::next_if_eq`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if_eq
Pkgsrc changes: * Add support for the big-endian arm64 NetBSD target (aarch64_be). * On NetBSD/i386, use the i586 (pentium) bootstrap kit variant in preference to i686. * Adjust patches, re-compute line offsets, re-compute crate checksums. * Remove a patch which was either integrated upstream and/or no longer applies. * Bump bootstraps to 1.50.0. * Move conditionals until after bsd.prefs.mk so that they work... * Default to "dist" build target if cross-compiling, but allow also to override via rust.BUILD_TARGET. * Allow overriding MAKE_JOBS_SAFE via rust.MAKE_JOBS_SAFE if you want a different trade-off between occasional breakage and performance. * Adjust platform.mk according to work already done in wip/rust/ * Add a patch to optimize the install.sh script used to install binary bootstraps to not do so many forks; use case/esac and parameter expansion instead of grep, sed and cut. * Drop building documentation for the binary bootstrap kits. This will also impact the lang/rust-bin package. For full documentation, build or install lang/rust as a package. Upstream changes: Version 1.51.0 (2021-03-25) ============================ Language -------- - [You can now parameterize items such as functions, traits, and `struct`s by constant values in addition to by types and lifetimes.][79135] Also known as "const generics" E.g. you can now write the following. Note: Only values of primitive integers, `bool`, or `char` types are currently permitted. ```rust struct GenericArray<T, const LENGTH: usize> { inner: [T; LENGTH] } impl<T, const LENGTH: usize> GenericArray<T, LENGTH> { const fn last(&self) -> Option<&T> { if LENGTH == 0 { None } else { Some(&self.inner[LENGTH - 1]) } } } ``` Compiler -------- - [Added the `-Csplit-debuginfo` codegen option for macOS platforms.][79570] This option controls whether debug information is split across multiple files or packed into a single file. **Note** This option is unstable on other platforms. - [Added tier 3\* support for `aarch64_be-unknown-linux-gnu`, `aarch64-unknown-linux-gnu_ilp32`, and `aarch64_be-unknown-linux-gnu_ilp32` targets.][81455] - [Added tier 3 support for `i386-unknown-linux-gnu` and `i486-unknown-linux-gnu` targets.][80662] - [The `target-cpu=native` option will now detect individual features of CPUs.][80749] \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [`Box::downcast` is now also implemented for any `dyn Any + Send + Sync` object.][80945] - [`str` now implements `AsMut<str>`.][80279] - [`u64` and `u128` now implement `From<char>`.][79502] - [`Error` is now implemented for `&T` where `T` implements `Error`.][75180] - [`Poll::{map_ok, map_err}` are now implemented for `Poll<Option<Result<T,E>>>`.][80968] - [`unsigned_abs` is now implemented for all signed integer types.][80959] - [`io::Empty` now implements `io::Seek`.][78044] - [`rc::Weak<T>` and `sync::Weak<T>`'s methods such as `as_ptr` are now implemented for `T: ?Sized` types.][80764] - [`Div` and `Rem` by their `NonZero` variant is now implemented for all unsigned integers.][79134] Stabilized APIs --------------- - [`Arc::decrement_strong_count`] - [`Arc::increment_strong_count`] - [`Once::call_once_force`] - [`Peekable::next_if_eq`] - [`Peekable::next_if`] - [`Seek::stream_position`] - [`array::IntoIter`] - [`panic::panic_any`] - [`ptr::addr_of!`] - [`ptr::addr_of_mut!`] - [`slice::fill_with`] - [`slice::split_inclusive_mut`] - [`slice::split_inclusive`] - [`slice::strip_prefix`] - [`slice::strip_suffix`] - [`str::split_inclusive`] - [`sync::OnceState`] - [`task::Wake`] - [`VecDeque::range`] - [`VecDeque::range_mut`] Cargo ----- - [Added the `split-debuginfo` profile option to control the -Csplit-debuginfo codegen option.][cargo/9112] - [Added the `resolver` field to `Cargo.toml` to enable the new feature resolver and CLI option behavior.][cargo/8997] Version 2 of the feature resolver will try to avoid unifying features of dependencies where that unification could be unwanted. Such as using the same dependency with a `std` feature in a build scripts and proc-macros, while using the `no-std` feature in the final binary. See the [Cargo book documentation][feature-resolver@2.0] for more information on the feature. Rustdoc ------- - [Rustdoc will now include documentation for methods available from _nested_ `Deref` traits.][80653] - [You can now provide a `--default-theme` flag which sets the default theme to use for documentation.][79642] Various improvements to intra-doc links: - [You can link to non-path primitives such as `slice`.][80181] - [You can link to associated items.][74489] - [You can now include generic parameters when linking to items, like `Vec<T>`.][76934] Misc ---- - [You can now pass `--include-ignored` to tests (e.g. with `cargo test -- --include-ignored`) to include testing tests marked `#[ignore]`.][80053] Compatibility Notes ------------------- - [WASI platforms no longer use the `wasm-bindgen` ABI, and instead use the wasm32 ABI.][79998] - [`rustc` no longer promotes division, modulo and indexing operations to `const` that could fail.][80579] - [The minimum version of glibc for the following platforms has been bumped to version 2.31 for the distributed artifacts.][81521] - `armv5te-unknown-linux-gnueabi` - `sparc64-unknown-linux-gnu` - `thumbv7neon-unknown-linux-gnueabihf` - `armv7-unknown-linux-gnueabi` - `x86_64-unknown-linux-gnux32` - [`atomic::spin_loop_hint` has been deprecated.][80966] It's recommended to use `hint::spin_loop` instead. Internal Only ------------- - [Consistently avoid constructing optimized MIR when not doing codegen][80718] [79135]: rust-lang/rust#79135 [74489]: rust-lang/rust#74489 [76934]: rust-lang/rust#76934 [79570]: rust-lang/rust#79570 [80181]: rust-lang/rust#80181 [79642]: rust-lang/rust#79642 [80945]: rust-lang/rust#80945 [80279]: rust-lang/rust#80279 [80053]: rust-lang/rust#80053 [79502]: rust-lang/rust#79502 [75180]: rust-lang/rust#75180 [79135]: rust-lang/rust#79135 [81521]: rust-lang/rust#81521 [80968]: rust-lang/rust#80968 [80959]: rust-lang/rust#80959 [80718]: rust-lang/rust#80718 [80653]: rust-lang/rust#80653 [80579]: rust-lang/rust#80579 [79998]: rust-lang/rust#79998 [78044]: rust-lang/rust#78044 [81455]: rust-lang/rust#81455 [80764]: rust-lang/rust#80764 [80749]: rust-lang/rust#80749 [80662]: rust-lang/rust#80662 [79134]: rust-lang/rust#79134 [80966]: rust-lang/rust#80966 [cargo/8997]: rust-lang/cargo#8997 [cargo/9112]: rust-lang/cargo#9112 [feature-resolver@2.0]: https://doc.rust-lang.org/nightly/cargo/reference/features.html#feature-resolver-version-2 [`Once::call_once_force`]: https://doc.rust-lang.org/stable/std/sync/struct.Once.html#method.call_once_force [`sync::OnceState`]: https://doc.rust-lang.org/stable/std/sync/struct.OnceState.html [`panic::panic_any`]: https://doc.rust-lang.org/stable/std/panic/fn.panic_any.html [`slice::strip_prefix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`slice::strip_suffix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`Arc::increment_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.increment_strong_count [`Arc::decrement_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.decrement_strong_count [`slice::fill_with`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.fill_with [`ptr::addr_of!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of.html [`ptr::addr_of_mut!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of_mut.html [`array::IntoIter`]: https://doc.rust-lang.org/nightly/std/array/struct.IntoIter.html [`slice::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive [`slice::split_inclusive_mut`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive_mut [`str::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.str.html#method.split_inclusive [`task::Wake`]: https://doc.rust-lang.org/nightly/std/task/trait.Wake.html [`Seek::stream_position`]: https://doc.rust-lang.org/nightly/std/io/trait.Seek.html#method.stream_position [`Peekable::next_if`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if [`Peekable::next_if_eq`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if_eq [`VecDeque::range`]: https://doc.rust-lang.org/nightly/std/collections/struct.VecDeque.html#method.range [`VecDeque::range_mut`]: https://doc.rust-lang.org/nightly/std/collections/struct.VecDeque.html#method.range_mut
This commit adds a new
split-debuginfo
option to Cargo compilationprofiles which gets forwarded to the
-Csplit-debuginfo
codegen optionin rustc. This commit also sets the default, only on macOS, to be
-Csplit-debuginfo=unpacked
. The purpose of this change is to leveragerust-lang/rust#79570 to avoid running
dsymutil
on incremental buildswhile also preserving a pleasant debugging experience by default. This
should lead to much faster incremental build times on macOS since
dsymutil
isn't exactly the speediest tool in the world.This is technically a breaking change in Cargo because we're no longer
by-default producing the
*.dSYM
folders on macOS. If those are stilldesired, however, authors can always run
dsymutil
themselves orotherwise configure
split-debuginfo = 'packed'
in theirmanifest/profile configuration.
EDIT (ehuss): The default has not changed, that may come in a later release, see below.