Enable -Zshare-generics for inline(never) functions

[Mark-Simulacrum]

This avoids inlining cross-crate generic items when possible that are
already marked inline(never), implying that the author is not intending
for the function to be inlined by callers. As such, having a local copy
may make it easier for LLVM to optimize but mostly just adds to binary
bloat and codegen time. In practice our benchmarks indicate this is
indeed a win for larger compilations, where the extra cost in dynamic
linking to these symbols is diminished compared to the advantages in
fewer copies that need optimizing in each binary.

It might also make sense it expand this with other heuristics (e.g.,
#[cold]) in the future, but this seems like a good starting point.

FWIW, I expect that doing cleanup in where we make the decision
what should/shouldn't be shared is also a good idea. Way too
much code needed to be tweaked to check this. But I'm hoping
to leave that for a follow-up PR rather than blocking this on it.

[Mark-Simulacrum]

@bors try @rust-timer queue

[bors]

⌛ Trying commit 5702d83 with merge 1f2a5ec...

[bors]

☀️ Try build successful - checks-actions
Build commit: 1f2a5ec (1f2a5ecd17d6b0415946e70d192b88b566dc73f8)

[rust-timer]

Finished benchmarking commit (1f2a5ec): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.4%	[0.3%, 18.5%]	203
Regressions ❌ (secondary)	2.7%	[0.3%, 16.2%]	207
Improvements ✅ (primary)	-1.2%	[-4.1%, -0.3%]	28
Improvements ✅ (secondary)	-3.9%	[-6.8%, -2.7%]	4
All ❌✅ (primary)	1.1%	[-4.1%, 18.5%]	231

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.5%	[0.6%, 3.9%]	16
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.7%	[-4.5%, -1.0%]	3
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.7%	[-4.5%, 3.9%]	19

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.6%	[0.9%, 11.5%]	73
Regressions ❌ (secondary)	4.5%	[1.2%, 14.3%]	119
Improvements ✅ (primary)	-1.8%	[-3.8%, -0.9%]	12
Improvements ✅ (secondary)	-4.4%	[-7.8%, -2.7%]	4
All ❌✅ (primary)	2.0%	[-3.8%, 11.5%]	85

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.0%, 0.3%]	15
Regressions ❌ (secondary)	0.2%	[0.0%, 0.3%]	38
Improvements ✅ (primary)	-0.7%	[-2.3%, -0.0%]	77
Improvements ✅ (secondary)	-8.7%	[-18.9%, -0.3%]	14
All ❌✅ (primary)	-0.6%	[-2.3%, 0.3%]	92

Bootstrap: 667.994s -> 656.525s (-1.72%)
Artifact size: 315.79 MiB -> 314.40 MiB (-0.44%)

[Mark-Simulacrum]

It looks like the majority of the additional cost comes from additional indirection in calls to standard library functions, e.g., a diff like this:

-  f61064:       e8 73 f4 49 03          call   44004dc <_RINvNtCsavh3npScQaX_5alloc7raw_vec11finish_growNtNtB4_5alloc6GlobalECs1UKmS5rlRwk_21rustc_trait_selection>
+  ffe57a:       ff 15 a0 24 03 03       call   *0x30324a0(%rip)        # 4030a20 <_ZN5alloc7raw_vec11finish_grow17h78aea5cebcfaa28aE@Base>

This means more work, particularly for short-lived compilations, since the symbol needs to get resolved at runtime now. That cost should be eliminated with #122362, which might take some time to land but is making progress now. Most downstream programs also don't pay it since they're not linking std dynamically.

A little of the extra cost from there seems to be due to these non-inlined functions now being codegen'd with frame pointers (due to #122646). That doesn't seem like something worth worrying about, we accepted some regression from those changes already.

My sense is that something like this is probably still a good idea despite the regressions. We do see good improvements in binary sizes (including >1MB of librustc_driver.so), and bootstrap times are significantly reduced. That suggests that this is a pretty good win for the larger crates while being a slight loss for smaller crates (instruction count timings are down for some of the larger primary crates as well, e.g., ripgrep and cranelift). That seems consistent with the loss due to additional indirection due to librustc_driver dynamically linking with the standard library.

Going to mark as ready for review as such.

r? compiler

[fmease]

r? compiler

[Mark-Simulacrum]

cc #14527

[Mark-Simulacrum]

Poking @wesleywiser as it's been ~3 weeks here.

[saethlin]

It's been another 3 weeks, and this looks really interesting.

r? saethlin

[saethlin]

Is this testing what we want? I was expecting to see a cross-crate call to a #[inline(never)] generic function in a test, because I think the point of this PR is to change the behavior for such calls, right?

[Mark-Simulacrum]

I think this is indeed checking that behavior, though it's rather obscure. As per the comment just above ("These should not contribute...") we are implicitly checking that these functions are absent in the mono-items of the downstream crate (tests/codegen-units/partitioning/extern-generic.rs) since they're not listed as MONO-ITEM declarations in that file. They are referenced though via the foo function above which has to get codegen'd in the downstream crate since it is generic and never called in this one.

I suppose we could try to write a more direct test case? But this also tests that the property holds transitively (i.e., we don't codegen the called function twice with inline(never) even if it's not directly being called).

IIRC, before my changes, this would fail.

[saethlin]

Oh I see! This is very interconnected, and I should really be used to that by now.

[saethlin]

Also I checked and yes the test does fail without the rest of this PR.

[Mark-Simulacrum]

Last perf run in #123244 (comment) continues to look pretty reasonable to me. We see some regressions (as expected), but primarily those are in smaller benchmarks. For larger benchmarks, e.g., the compiler crates, we see sizeable improvements in wall times and binary sizes. It seems likely this still remains a good idea.

@saethlin, you previously already approved this, so probably makes sense for you to take another look? The core changes should be the same as before (many months ago :)

@rustbot ready

[saethlin]

Heh, I was already looking at this again because I'm quite excited to land it.

@rustbot label: +perf-regression-triaged
The binary size improvements are quite nice, and the PR has mixed icount changes, but all the regressions over 0.5% are to helloworld. Many but not all of these tiny regressions seem related to green queries. I suspect there's one or two functions in the query system that are #[inline(never)] but were getting some useful optimizations from having internal linkage. I think root-causing the regressions further is more trouble than it's worth, given all the other compile time improvement opportunities we have.

@bors r+

[bors]

📌 Commit 2a0fcc7 has been approved by saethlin

It is now in the queue for this repository.

[bors]

⌛ Testing commit 2a0fcc7 with merge b773e21...

[bors]

💔 Test failed - checks-actions

[Mark-Simulacrum]

Some sort of randomness with the begin_panic symbol, seemingly for the same builder sometimes showing begin_panic::<&str> and sometimes begin_panic. I'm pretty confused because AFAICT, they should have identical CI configurations. Maybe we're running into pre-existing instability/non-determinism with -Csymbol-mangling-version and -Zshare-generics?

[Mark-Simulacrum]

OK, the issue is that if std is built with v0 mangling, then the test output differs. That only happens in PR CI today (due to

rust/src/ci/github-actions/jobs.yml

Line 101 in 6b6a867

ENABLE_GCC_CODEGEN: "1"

), I started a Zulip thread about that difference.

For this PR I'll probably just normalize away the difference in the small number of affected tests.

[Mark-Simulacrum]

@bors r=saethlin

[bors]

📌 Commit efc4eb3 has been approved by saethlin

It is now in the queue for this repository.