Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use precalculated dominators in explain_borrow #65172

Merged

Conversation

tanriol
Copy link
Contributor

@tanriol tanriol commented Oct 6, 2019

This looks like the only place calculating dominators from the MIR body every time instead of using the ones stored on the MirBorrowckCtxt. For example, in #65131 a big generated function with a number of borrowck errors takes a few hours(!) recalculating the dominators while explaining the errors.

I don't know enough about this part of rustc codebase to know for sure that this change is correct, but no tests seem to fail as a result of this change in local testing.

This looks like the only place calculating dominators from the MIR body every time instead of using the ones stored on the `MirBorrowckCtxt`. For example, in rust-lang#65131 a big generated function with a number of borrowck errors takes a few hours(!) recalculating the dominators while explaining the errors.

I don't know enough about this part of rustc codebase to know for sure that this change is correct, but no tests seem to fail as a result of this change in local testing.
@rust-highfive
Copy link
Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @eddyb (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 6, 2019
@nagisa
Copy link
Member

nagisa commented Oct 7, 2019

@bors r+

The only case where this change can be wrong is in case we have wrong code elsewhere that does not invalidate the cache in some way after doing some modification. Since borrowck does not mutate anything, this change seems perfectly fine to me.

@bors
Copy link
Contributor

bors commented Oct 7, 2019

📌 Commit 4cc707c has been approved by nagisa

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 7, 2019
@nagisa
Copy link
Member

nagisa commented Oct 7, 2019

@bors r- delegate+

@tanriol would be great if you also added a regression test case. A good test could be just something that runs for a dozen minutes without this fix and completes in under a second with the fix. If that is not feasible, then something that checks whether the compiler terminates with the minimized reproducer from #65131 would also do.

@bors
Copy link
Contributor

bors commented Oct 7, 2019

✌️ @tanriol can now approve this pull request

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Oct 7, 2019
@bors
Copy link
Contributor

bors commented Oct 7, 2019

✌️ @tanriol can now approve this pull request

@nagisa
Copy link
Member

nagisa commented Oct 7, 2019

I have also added Fixes #65131 to the issue description body for you.

@kpp
Copy link
Contributor

kpp commented Oct 7, 2019

On this case: #65131 (comment)

19.8s (nightly 1.40) vs 0.3s (this branch)

<3

@tanriol
Copy link
Contributor Author

tanriol commented Oct 7, 2019

@nagisa What's the proper way of testing this? As written currently this is a performance-only change, and compiletest does not seem to have any test timeouts. Sure, I can scale up this test to take an hour and hope there are CI timeouts that'll fail, but this does not seem to be a good idea.

The options I'm seeing are:

  • work on adding timeout support to compiletest,
  • (probably easier) add to compiletest support for success time limit without terminating rustc,
  • make a run-make test and run rustc with a timeout from the Makefile (probably UNIX-only),
  • (alternatively) add some checks to rustc to ensure that body.dominators() is not called "too many" times,
  • any other ideas?

UPD: sorry, this reply was written based on the email notification without noticing the edits.

@glandium
Copy link
Contributor

glandium commented Oct 7, 2019

How about adding failure tests on https://perf.rust-lang.org/ ?

@nagisa
Copy link
Member

nagisa commented Oct 7, 2019

The easiest approach is probably a run-pass test similar to this one which spawns itself as another process with an extra argument and then calculates the time necessary for the child to complete from the parent process.

@tanriol
Copy link
Contributor Author

tanriol commented Oct 7, 2019

The slowdown is not to runtime, but to compilation process, so the alternative would be to... spawn rustc, I guess?

@bjorn3
Copy link
Member

bjorn3 commented Oct 7, 2019

Useful trick: When you use - as source file, rustc will read from stdin. Just make sure you close stdin, as rustc will otherwise keep waiting for more input.

@nagisa
Copy link
Member

nagisa commented Oct 7, 2019

The slowdown is not to runtime, but to compilation process

Ah yeah, sorry, I didn’t wake up entirely before writing that down. In that case this one is an example of a test that invokes rustc within some arbitrary Rust executable.

@JohnCSimon JohnCSimon added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 12, 2019
@JohnCSimon
Copy link
Member

Hello from triage - this has sat idle for the last 5 days.
@eddyb I can't tell if this is waiting to be merged or additional tests from @tanriol
CC: @spastorino

@spastorino
Copy link
Member

Seemed to be waiting for additional tests. Looks good to me btw :).

@tanriol
Copy link
Contributor Author

tanriol commented Oct 12, 2019

Going to add a test in a couple days.

@tmandry
Copy link
Member

tmandry commented Oct 15, 2019

I'm confused why we would add this test in-tree instead of as a benchmark to rustc-perf. (Example PR, btw)

@tanriol
Copy link
Contributor Author

tanriol commented Oct 16, 2019

@tmandry Thank you for the example.

Is rustc-perf a correct place for, basically, a regression test for error reporting performance in a very specific case of a huge generated function with borrow-check errors? It feels to me a little heavyweight for such a small problem (cluttering of the "all benchmarks" view with an additional graph, for example).

Basically, my feeling was that rustc-perf / perf.rust-lang.org is a human-oriented resource, not an automated testing oriented one, and the exact performance of this use case is not something worth permanent human attention. However, if you consider this position wrong, please tell me so and I'll add the test there.

@nagisa
Copy link
Member

nagisa commented Oct 16, 2019

rustc perf tests will not block landing and will only test retroactively, which is not terrible, but not ideal for issues like these either.

That being said glandium suggests there are some sort of a "failure test" mode in rustc-perf? In that case it sounds like a fairly good compromise.

@nagisa
Copy link
Member

nagisa commented Oct 16, 2019

@bors r+

I am approving this PR for landing due to the large improvement on perf that this gives, with an expectation that a regression test is added as a follow-up PR (either to rustc itself or to rustc-perf or somewhere else entirely?)

To track this follow-up work, I am a) removing the Fixes #65131 annotation so that the issue does not get closed and tagging the issue with E-needs-test.

@bors
Copy link
Contributor

bors commented Oct 16, 2019

📌 Commit 4cc707c has been approved by nagisa

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 16, 2019
@bors
Copy link
Contributor

bors commented Oct 16, 2019

💡 This pull request was already approved, no need to approve it again.

@bors
Copy link
Contributor

bors commented Oct 16, 2019

📌 Commit 4cc707c has been approved by nagisa

@bors
Copy link
Contributor

bors commented Oct 16, 2019

⌛ Testing commit 4cc707c with merge 53aca55...

bors added a commit that referenced this pull request Oct 16, 2019
… r=nagisa

use precalculated dominators in explain_borrow

This looks like the only place calculating dominators from the MIR body every time instead of using the ones stored on the `MirBorrowckCtxt`. For example, in #65131 a big generated function with a number of borrowck errors takes a few hours(!) recalculating the dominators while explaining the errors.

I don't know enough about this part of rustc codebase to know for sure that this change is correct, but no tests seem to fail as a result of this change in local testing.
@bors
Copy link
Contributor

bors commented Oct 16, 2019

☀️ Test successful - checks-azure
Approved by: nagisa
Pushing 53aca55 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Oct 16, 2019
@bors bors merged commit 4cc707c into rust-lang:master Oct 16, 2019
@nagisa nagisa added the beta-nominated Nominated for backporting to the compiler in the beta channel. label Oct 17, 2019
@pnkfelix
Copy link
Member

Discussed at T-compiler meeting. Accepted for beta backport.

@pnkfelix pnkfelix added the beta-accepted Accepted for backporting to the compiler in the beta channel. label Oct 17, 2019
@Mark-Simulacrum Mark-Simulacrum removed the beta-nominated Nominated for backporting to the compiler in the beta channel. label Oct 22, 2019
bors added a commit that referenced this pull request Oct 24, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
bors added a commit that referenced this pull request Oct 26, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
 *  [beta] Revert "Auto merge of #62948 - matklad:failable-file-loading, r=petro… #65273
 *  save-analysis: Don't ICE when resolving qualified type paths in struct members #65353
 *  save-analysis: Nest tables when processing impl block definitions #65511
 *  Avoid ICE when checking `Destination` of `break` inside a closure #65518
 *  Avoid ICE when adjusting bad self ty #65755
 *  workaround msys2 bug #65762
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beta-accepted Accepted for backporting to the compiler in the beta channel. merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.