Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

typeck in parallel #113519

Merged
merged 1 commit into from
Jul 14, 2023
Merged

typeck in parallel #113519

merged 1 commit into from
Jul 14, 2023

Conversation

SparrowLii
Copy link
Member

#108118 caused typeck to be transferred to the serial part (check_unused), which made the performance of parallel rustc significantly reduced.

This pr re-parallelize this part, which increases the average performance improvement of parallel rustc in full and incr-full scenarios from 14.4% to 23.2%.

r? @cjgillot
cc @oli-obk @Zoxc

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jul 10, 2023
@SparrowLii
Copy link
Member Author

Because AonoConsts also belong to body_owners, but they should be fed in their parents' typeck, so we skip them.
According to @oli-obk, we would creat the DefIds for AonoConsts in their parents' typeck in the future. We could get rid of this patch and directly parallelize the iterators in other processes then.

Copy link
Contributor

@cjgillot cjgillot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This same function iterates over modules serially. Should those call some par_for_each_module method?
Probably to be decided by looking at profiles.

compiler/rustc_hir_analysis/src/lib.rs Outdated Show resolved Hide resolved
// FIXME: Parallelize other iterators(e.g. `body_owners` in `check_unused`) instead
// when we implement creating `DefId`s for anon constants during their parents' typeck.
// Since doing so will produce query cycle errors at current.
if cfg!(parallel_compiler) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we should have different logic for serial vs parallel compiler.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it doesn't change the logic, because check_unused will call all typeck and fetch the used_trait_impl information in the results. I moved this part into check_unused to make it look more reasonable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code really belongs in check_crate, and not in check_unused. My suggestion was just to remove the if cfg!(parallel_compiler), and run this unconditionally.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reverts part of #108118. I agree this is reasonable.

Copy link
Member Author

@SparrowLii SparrowLii Jul 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed cfg!(parallel_compiler) and currented some ui tests, since the order of errors during typeck is changed

@@ -494,6 +494,18 @@ pub fn check_crate(tcx: TyCtxt<'_>) -> Result<(), ErrorGuaranteed> {
tcx.hir().for_each_module(|module| tcx.ensure().check_mod_item_types(module))
});

// FIXME: Parallelize other iterators(e.g. `body_owners` in `check_unused`) instead
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this sentence. Should the FIXME be in `check_unused?

Copy link
Member Author

@SparrowLii SparrowLii Jul 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I moved this part into check_unused to make it look more reasonable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just add a FIXME above check_unused now

@SparrowLii
Copy link
Member Author

SparrowLii commented Jul 10, 2023

Thanks for reviewing! If you mean item_types_checking, It has little effect on the profile so I didn't modify it.

@rust-log-analyzer

This comment has been minimized.

@SparrowLii
Copy link
Member Author

SparrowLii commented Jul 11, 2023

I don't understand why the clippy test failed suddenly when I fixed the comment

@rust-log-analyzer

This comment has been minimized.

@oli-obk
Copy link
Contributor

oli-obk commented Jul 11, 2023

That's a spurious CI bug that I am looking into.

@cjgillot
Copy link
Contributor

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 11, 2023
@bors
Copy link
Contributor

bors commented Jul 11, 2023

⌛ Trying commit 50896c1 with merge 96389d030446cf91feda468c7b2c7a10ff67e35b...

@bors
Copy link
Contributor

bors commented Jul 11, 2023

☀️ Try build successful - checks-actions
Build commit: 96389d030446cf91feda468c7b2c7a10ff67e35b (96389d030446cf91feda468c7b2c7a10ff67e35b)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (96389d030446cf91feda468c7b2c7a10ff67e35b): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
3.5% [3.5%, 3.5%] 1
Regressions ❌
(secondary)
3.8% [3.1%, 4.5%] 2
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-3.0% [-3.0%, -3.0%] 1
All ❌✅ (primary) 3.5% [3.5%, 3.5%] 1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 656.645s -> 656.709s (0.01%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 11, 2023
@cjgillot
Copy link
Contributor

@bors r+

@bors
Copy link
Contributor

bors commented Jul 13, 2023

📌 Commit 50896c1 has been approved by cjgillot

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 13, 2023
@bors
Copy link
Contributor

bors commented Jul 14, 2023

⌛ Testing commit 50896c1 with merge 7d60819...

@bors
Copy link
Contributor

bors commented Jul 14, 2023

☀️ Test successful - checks-actions
Approved by: cjgillot
Pushing 7d60819 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jul 14, 2023
@bors bors merged commit 7d60819 into rust-lang:master Jul 14, 2023
@rustbot rustbot added this to the 1.73.0 milestone Jul 14, 2023
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (7d60819): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
3.4% [3.0%, 4.0%] 5
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 659.456s -> 656.192s (-0.49%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants