Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix variable debuginfo being optimized away at mir-opt-level=2 #103657

Closed

Conversation

wesleywiser
Copy link
Member

@wesleywiser wesleywiser commented Oct 27, 2022

The DeadStoreElmination pass was removing the uses of these locals
because the are never used in the MIR body except for debuginfo. I've
updated the pass to consider locals referenced by debuginfo as always
alive unless a higher MIR opt level is requested by the user.

Fixes #103655

r? @ghost

The `DeadStoreElmination` pass was removing the uses of these locals
because the are never used in the MIR body except for debuginfo. I've
updated the pass to consider locals referenced by debuginfo as always
alive unless a higher MIR opt level is requested by the user.
@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Oct 27, 2022
@rustbot
Copy link
Collaborator

rustbot commented Oct 27, 2022

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

@JakobDegen
Copy link
Contributor

According to rust-lang/compiler-team#319 , mir opt level 2 is not required to maintain debuggability. Has this changed?

@rust-log-analyzer

This comment has been minimized.

@wesleywiser
Copy link
Member Author

According to rust-lang/compiler-team#319 , mir opt level 2 is not required to maintain debuggability. Has this changed?

It changed in #82736 which was merged after that policy. I looked briefly but don't see our policy actually documented anywhere other than that closed MCP. I might take a stab at introducing an enum for mir-opt-level that maps the 0-4 levels to what they actually mean but first I want to see what kind of impact this has on compile times.

@wesleywiser
Copy link
Member Author

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Oct 27, 2022
@bors
Copy link
Contributor

bors commented Oct 27, 2022

⌛ Trying commit daf2968 with merge ba1d739bab63e852f8d36184b10d102e02ac3113...

@JakobDegen
Copy link
Contributor

I'm incredibly confused. When I asked about the policy on Zulip no one could seem to agree what it was, and now this PR is proposing another option. Certainly if this is right then lots of my commits to the repo have been wrong (although I'm not the only one). I also have some thoughts about a mir opt level policy that doesn't allow us to regress debuginfo on any opt level that users will realistically be turning on.

It seems like we need a new MCP on this topic, given that the previous one does not seem to have resolved this topic

@wesleywiser
Copy link
Member Author

wesleywiser commented Oct 27, 2022

That's a fair concern! I agree, the current situation is very confusing, probably because we never really documented it anywhere and then it was later modified without a clear explanation of the new level being introduced. Sorting this out via an MCP and then actually documenting it somewhere seems like a reasonable next step.

At the same time, it seems like a bug to me that disabling MIR optimizations for code like that of the test gives you both better debuginfo and doesn't hurt code quality either (LLVM is able to perform this optimization just as well as we can but they do it while preserving debuginfo). If there isn't much of a compile time impact from this change, I think we should merge it.

@JakobDegen
Copy link
Contributor

Sure, I don't feel particularly strongly about this PR specifically (although that may change in the future). That being said, do we know what is needed in order for us to fix debuginfo in the same way LLVM does?

@wesleywiser
Copy link
Member Author

I also have some thoughts about a mir opt level policy that doesn't allow us to regress debuginfo on any opt level that users will realistically be turning on.

Yet another option (sorry), would be to tie the presence of debuginfo in MIR to -Cdebuginfo=2. If the user isn't actually requesting debuginfo, I don't see a strong reason we should retain it in MIR. At the same time, if the user is specifically asking for debuginfo, then we should put some effort into not breaking it.

@wesleywiser
Copy link
Member Author

wesleywiser commented Oct 27, 2022

Sure, I don't feel particularly strongly about this PR specifically (although that may change in the future). That being said, do we know what is needed in order for us to fix debuginfo in the same way LLVM does?

I think I put most of the infrastructure in place to support this in #73210 but it wasn't enabled because it wasn't working correctly on Windows. At the time, I knew nothing about Windows debuginfo so perhaps now I can resolve whatever that issue was. 😄

@JakobDegen
Copy link
Contributor

I've been thinking for a while that optimization "levels" might be a pretty bad solution to the problem they're trying to solve. Maybe this is a good opportunity to revisit this problem from the beginning, start by figuring out what the use cases are that we're trying to support, and design from there

@bors
Copy link
Contributor

bors commented Oct 28, 2022

☀️ Try build successful - checks-actions
Build commit: ba1d739bab63e852f8d36184b10d102e02ac3113 (ba1d739bab63e852f8d36184b10d102e02ac3113)

@rust-timer
Copy link
Collaborator

Queued ba1d739bab63e852f8d36184b10d102e02ac3113 with parent 0da281b, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (ba1d739bab63e852f8d36184b10d102e02ac3113): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean1 range count2
Regressions ❌
(primary)
0.5% [0.2%, 1.0%] 21
Regressions ❌
(secondary)
1.0% [0.2%, 3.3%] 22
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 0.5% [0.2%, 1.0%] 21

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean1 range count2
Regressions ❌
(primary)
2.2% [0.2%, 3.6%] 3
Regressions ❌
(secondary)
2.6% [2.4%, 2.9%] 2
Improvements ✅
(primary)
-3.1% [-3.3%, -2.9%] 2
Improvements ✅
(secondary)
-5.6% [-5.6%, -5.6%] 1
All ❌✅ (primary) 0.1% [-3.3%, 3.6%] 5

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean1 range count2
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
2.2% [2.1%, 2.3%] 5
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

Footnotes

  1. the arithmetic mean of the percent change 2 3

  2. number of relevant changes 2 3

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Oct 28, 2022
@wesleywiser
Copy link
Member Author

Just trying to see what we need to do to recover the throughput (if possible).

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Oct 28, 2022
@bors
Copy link
Contributor

bors commented Oct 28, 2022

⌛ Trying commit 51d5932 with merge 5552e8f2ea34ebec9325fbba2b896872124d77f4...

@rust-log-analyzer
Copy link
Collaborator

The job x86_64-gnu-llvm-13 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
 finished in 0.596 seconds
Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)

running 158 tests
....................................F.....F....FF.F........F............................ 88/158
Some tests failed in compiletest suite=incremental mode=incremental host=x86_64-unknown-linux-gnu target=x86_64-unknown-linux-gnu
failures:

---- [incremental] src/test/incremental/hashes/closure_expressions.rs stdout ----


error in revision `cfail2`: test compilation failed although it shouldn't!
status: exit status: 1
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/hashes/closure_expressions.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "cfail2" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/closure_expressions/closure_expressions.inc" "-Z" "incremental-verify-ich" "-O" "--error-format" "json" "--json" "future-incompat" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/closure_expressions" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "-O" "-Zincremental-ignore-spans" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/closure_expressions/auxiliary"
stdout: none
--- stderr -------------------------------
error: `optimized_mir(add_parameter)` should be dirty but is not
   |
LL | pub fn add_parameter() {
   | ^^^^^^^^^^^^^^^^^^^^^^


error: aborting due to previous error
------------------------------------------


---- [incremental] src/test/incremental/hashes/enum_constructors.rs stdout ----

error in revision `cfail2`: test compilation failed although it shouldn't!
status: exit status: 1
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/hashes/enum_constructors.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "cfail2" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/enum_constructors/enum_constructors.inc" "-Z" "incremental-verify-ich" "-O" "--error-format" "json" "--json" "future-incompat" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/enum_constructors" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "-O" "-Zincremental-ignore-spans" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/enum_constructors/auxiliary"
stdout: none
--- stderr -------------------------------
error: `optimized_mir(change_constructor_variant_c_like)` should be clean but is not
   |
   |
LL | pub fn change_constructor_variant_c_like() {

error: aborting due to previous error
------------------------------------------



---- [incremental] src/test/incremental/hashes/let_expressions.rs stdout ----

error in revision `cfail2`: test compilation failed although it shouldn't!
status: exit status: 1
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/hashes/let_expressions.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "cfail2" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/let_expressions/let_expressions.inc" "-Z" "incremental-verify-ich" "-O" "--error-format" "json" "--json" "future-incompat" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/let_expressions" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "-O" "-Zincremental-ignore-spans" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/let_expressions/auxiliary"
stdout: none
--- stderr -------------------------------
error: `optimized_mir(change_mutability_of_slot)` should be dirty but is not
   |
LL | pub fn change_mutability_of_slot() {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


error: `optimized_mir(change_mutability_of_binding_in_pattern)` should be dirty but is not
   |
LL | pub fn change_mutability_of_binding_in_pattern() {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


error: `optimized_mir(change_initializer)` should be clean but is not
   |
LL | pub fn change_initializer() {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^


error: aborting due to 3 previous errors
------------------------------------------


---- [incremental] src/test/incremental/hashes/for_loops.rs stdout ----

error in revision `cfail2`: test compilation failed although it shouldn't!
status: exit status: 1
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/hashes/for_loops.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "cfail2" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/for_loops/for_loops.inc" "-Z" "incremental-verify-ich" "-O" "--error-format" "json" "--json" "future-incompat" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/for_loops" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "-O" "-Zincremental-ignore-spans" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/for_loops/auxiliary"
stdout: none
--- stderr -------------------------------
error: `optimized_mir(change_loop_body)` should be clean but is not
   |
   |
LL | pub fn change_loop_body() {

error: aborting due to previous error
------------------------------------------



---- [incremental] src/test/incremental/hashes/loop_expressions.rs stdout ----

error in revision `cfail2`: test compilation failed although it shouldn't!
status: exit status: 1
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/hashes/loop_expressions.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "cfail2" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/loop_expressions/loop_expressions.inc" "-Z" "incremental-verify-ich" "-O" "--error-format" "json" "--json" "future-incompat" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/loop_expressions" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "-O" "-Zincremental-ignore-spans" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/loop_expressions/auxiliary"
stdout: none
--- stderr -------------------------------
error: `optimized_mir(change_loop_body)` should be clean but is not
   |
   |
LL | pub fn change_loop_body() {

error: aborting due to previous error
------------------------------------------



---- [incremental] src/test/incremental/hashes/while_loops.rs stdout ----

error in revision `cfail2`: test compilation failed although it shouldn't!
status: exit status: 1
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/hashes/while_loops.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "cfail2" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/while_loops/while_loops.inc" "-Z" "incremental-verify-ich" "-O" "--error-format" "json" "--json" "future-incompat" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/while_loops" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "-O" "-Zincremental-ignore-spans" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hashes/while_loops/auxiliary"
stdout: none
--- stderr -------------------------------
error: `optimized_mir(change_loop_body)` should be clean but is not
   |
   |
LL | pub fn change_loop_body() {


error: `optimized_mir(change_loop_condition)` should be clean but is not
   |
LL | pub fn change_loop_condition() {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

@bors
Copy link
Contributor

bors commented Oct 28, 2022

☀️ Try build successful - checks-actions
Build commit: 5552e8f2ea34ebec9325fbba2b896872124d77f4 (5552e8f2ea34ebec9325fbba2b896872124d77f4)

@rust-timer
Copy link
Collaborator

Queued 5552e8f2ea34ebec9325fbba2b896872124d77f4 with parent 5237c4d, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (5552e8f2ea34ebec9325fbba2b896872124d77f4): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.5% [0.3%, 1.6%] 13
Regressions ❌
(secondary)
0.6% [0.3%, 1.2%] 8
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 0.5% [0.3%, 1.6%] 13

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
3.9% [0.2%, 6.1%] 5
Regressions ❌
(secondary)
3.2% [3.2%, 3.2%] 1
Improvements ✅
(primary)
-4.0% [-4.0%, -4.0%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 2.5% [-4.0%, 6.1%] 6

Cycles

This benchmark run did not return any relevant results for this metric.

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Oct 28, 2022
@wesleywiser wesleywiser added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 8, 2022
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 23, 2023
Turn on ConstDebugInfo pass.

Split from rust-lang#103657

Moving those constant into debuginfo allows to shrink the number of locals and the actual size of the MIR body.
@JohnCSimon
Copy link
Member

@wesleywiser Can you please post your status on this PR? It has sat idle for months.

@workingjubilee workingjubilee added the A-debuginfo Area: Debugging information in compiled programs (DWARF, PDB, etc.) label Jul 30, 2023
@JohnCSimon
Copy link
Member

@wesleywiser

Ping from triage: I'm closing this due to inactivity, Please reopen when you are ready to continue with this.
Note: if you are going to continue please open the PR BEFORE you push to it, else you won't be able to reopen - this is a quirk of github.
Thanks for your contribution.

@rustbot label: +S-inactive

@JohnCSimon JohnCSimon closed this Dec 17, 2023
@rustbot rustbot added the S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs. label Dec 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-debuginfo Area: Debugging information in compiled programs (DWARF, PDB, etc.) perf-regression Performance regression. S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MIR optimizations at mir-opt-level=2 remove some debuginfo
8 participants