Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Also share drop-glue when compiling with -Zshare-generics (i.e. at opt-level=0) #68414

Merged
merged 5 commits into from
Jan 24, 2020

Conversation

michaelwoerister
Copy link
Member

This PR adds drop-glue to the set of monomorphizations that can be shared across crates via -Zshare-generics.

This version of the PR might have detrimental effects on performance as it makes lots of stuff dependent on a single query results (upstream_monomorphizations_for(def_id_of_drop_in_place)). That should be fixable but let's do a perf run first.

Potentially fixes issue #64140. (cc @alexcrichton)
The changes here are related to @matthewjasper's #67332 but should be mostly orthogonal.

r? @ghost

@michaelwoerister
Copy link
Member Author

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@bors
Copy link
Contributor

bors commented Jan 21, 2020

⌛ Trying commit 98034e9 with merge e9acaef...

bors added a commit that referenced this pull request Jan 21, 2020
Also share drop-glue when compiling with -Zshare-generics (i.e. at opt-level=0)

This PR adds drop-glue to the set of monomorphizations that can be shared across crates via `-Zshare-generics`.

This version of the PR might have detrimental effects on performance as it makes lots of stuff dependent on a single query results (`upstream_monomorphizations_for(def_id_of_drop_in_place)`). That should be fixable but let's do a perf run first.

Potentially fixes issue #64140. (cc @alexcrichton)
The changes here are related to @matthewjasper's #67332 but should be mostly orthogonal.

r? @ghost
@bors
Copy link
Contributor

bors commented Jan 21, 2020

☀️ Try build successful - checks-azure
Build commit: e9acaef (e9acaef63c64146d312512a5bfe30896d8c9c877)

@rust-timer
Copy link
Collaborator

Queued e9acaef with parent ce361fb, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit e9acaef, comparison URL.

@alexcrichton
Copy link
Member

Hard to get a better try run than that :)

r=me on the code here too

@michaelwoerister
Copy link
Member Author

Hm, I find the results a bit underwhelming actually. Let's see if having a separate query drop-glue improves things. If not, merging this simpler version is good too. There's some nice cleanup/documentation in there.

@michaelwoerister
Copy link
Member Author

@bors try @rust-timer queue

This adds another query layer for drop-glue so we get better granularity (an approach that could be trivially extend to similar cases like Clone impls?). Let's see what this does to performance.

Unfortunately I also found a bug. Running UI tests with optimize-tests = false gives 5 linker errors where drop-glue isn't found. Curiously the error goes away when switching to the new symbol mangling scheme. (I almost always use the new mangling scheme lately. It makes debugging certain things so much easier). I'll have to look into that.

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@bors
Copy link
Contributor

bors commented Jan 22, 2020

⌛ Trying commit 9494974 with merge d9ce420...

bors added a commit that referenced this pull request Jan 22, 2020
Also share drop-glue when compiling with -Zshare-generics (i.e. at opt-level=0)

This PR adds drop-glue to the set of monomorphizations that can be shared across crates via `-Zshare-generics`.

This version of the PR might have detrimental effects on performance as it makes lots of stuff dependent on a single query results (`upstream_monomorphizations_for(def_id_of_drop_in_place)`). That should be fixable but let's do a perf run first.

Potentially fixes issue #64140. (cc @alexcrichton)
The changes here are related to @matthewjasper's #67332 but should be mostly orthogonal.

r? @ghost
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-01-22T11:27:09.7167033Z ========================== Starting Command Output ===========================
2020-01-22T11:27:09.7168749Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/4a42801c-9c8a-428b-baec-28893d352f05.sh
2020-01-22T11:27:09.7168846Z 
2020-01-22T11:27:09.7172061Z ##[section]Finishing: Disable git automatic line ending conversion
2020-01-22T11:27:09.7180161Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/68414/merge to s
2020-01-22T11:27:09.7182177Z Task         : Get sources
2020-01-22T11:27:09.7182265Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-01-22T11:27:09.7182301Z Version      : 1.0.0
2020-01-22T11:27:09.7182336Z Author       : Microsoft
---
2020-01-22T11:27:10.7199941Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-01-22T11:27:10.7219682Z ##[command]git config gc.auto 0
2020-01-22T11:27:10.7221822Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-01-22T11:27:10.7224284Z ##[command]git config --get-all http.proxy
2020-01-22T11:27:10.7230682Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/68414/merge:refs/remotes/pull/68414/merge

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Jan 22, 2020

☀️ Try build successful - checks-azure
Build commit: d9ce420 (d9ce4204180bce8508f3a3401877ab044e24a632)

@rust-timer
Copy link
Collaborator

Queued d9ce420 with parent 2f688ac, future comparison URL.

@michaelwoerister
Copy link
Member Author

Alright, I found the bug. Turns out InstanceDef::Item(drop_in_place::<Foo>) and InstanceDef::DropGlue(Foo) can be used interchangeable, except for when using the legacy symbol mangling, where the two result in a slightly different symbol name. I'll push the fix shortly. The perf.rlo benchmarks are not affected apparently?

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit d9ce420, comparison URL.

@michaelwoerister
Copy link
Member Author

This is ready for an actual review.

@michaelwoerister michaelwoerister marked this pull request as ready for review January 23, 2020 12:20
@michaelwoerister
Copy link
Member Author

michaelwoerister commented Jan 23, 2020

r? @alexcrichton (d3ca81c96b758dd7ec89990f00b26282427117cd 7bbdeb6bcc32f77186ab87fca946f4461dee15a1 was added since you last looked at it)

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-01-23T12:16:25.0850564Z ========================== Starting Command Output ===========================
2020-01-23T12:16:25.0869994Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/549951b6-e030-4c80-aecf-1284b381825a.sh
2020-01-23T12:16:25.1159598Z 
2020-01-23T12:16:25.1196015Z ##[section]Finishing: Disable git automatic line ending conversion
2020-01-23T12:16:25.1200424Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/68414/merge to s
2020-01-23T12:16:25.1202442Z Task         : Get sources
2020-01-23T12:16:25.1202475Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-01-23T12:16:25.1202524Z Version      : 1.0.0
2020-01-23T12:16:25.1202556Z Author       : Microsoft
---
2020-01-23T12:16:26.1911799Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-01-23T12:16:26.1927213Z ##[command]git config gc.auto 0
2020-01-23T12:16:26.1929268Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-01-23T12:16:26.1932793Z ##[command]git config --get-all http.proxy
2020-01-23T12:16:26.1940602Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/68414/merge:refs/remotes/pull/68414/merge
---
2020-01-23T13:12:41.1588140Z .................................................................................................... 1700/9546
2020-01-23T13:12:47.2041682Z .................................................................................................... 1800/9546
2020-01-23T13:12:58.5362847Z .....................i.............................................................................. 1900/9546
2020-01-23T13:13:05.4565838Z .................................................................................................... 2000/9546
2020-01-23T13:13:20.2692724Z ...........iiiii.................................................................................... 2100/9546
2020-01-23T13:13:29.6023814Z .................................................................................................... 2300/9546
2020-01-23T13:13:31.9301145Z .................................................................................................... 2400/9546
2020-01-23T13:13:37.1323436Z .................................................................................................... 2500/9546
2020-01-23T13:13:57.3284204Z .................................................................................................... 2600/9546
---
2020-01-23T13:16:33.9019137Z .......................................................i...............i............................ 4900/9546
2020-01-23T13:16:41.7527759Z .................................................................................................... 5000/9546
2020-01-23T13:16:49.5655341Z ..................................................................................................i. 5100/9546
2020-01-23T13:16:54.5295844Z .................................................................................................... 5200/9546
2020-01-23T13:17:05.1439207Z ......................................................................ii.ii...........i............. 5300/9546
2020-01-23T13:17:13.8844393Z .......i............................................................................................ 5500/9546
2020-01-23T13:17:23.7292975Z .................................................................................................... 5600/9546
2020-01-23T13:17:30.1607313Z ........................................................i........................................... 5700/9546
2020-01-23T13:17:36.9750359Z .................................................................................................... 5800/9546
2020-01-23T13:17:36.9750359Z .................................................................................................... 5800/9546
2020-01-23T13:17:46.6254005Z .................................................................................................... 5900/9546
2020-01-23T13:17:53.4705722Z ...............................................ii...i..ii...........i............................... 6000/9546
2020-01-23T13:18:15.4155577Z .................................................................................................... 6200/9546
2020-01-23T13:18:23.5970809Z .................................................................................................... 6300/9546
2020-01-23T13:18:23.5970809Z .................................................................................................... 6300/9546
2020-01-23T13:18:29.3497781Z ...........................................................................i..ii.................... 6400/9546
2020-01-23T13:18:55.7847014Z .................................................................................................... 6600/9546
2020-01-23T13:19:00.4116075Z ...................................................i................................................ 6700/9546
2020-01-23T13:19:02.5304545Z .................................................................................................... 6800/9546
2020-01-23T13:19:04.7133199Z ..................................................i................................................. 6900/9546
---
2020-01-23T13:20:44.9434786Z .................................................................................................... 7600/9546
2020-01-23T13:20:50.7234890Z .................................................................................................... 7700/9546
2020-01-23T13:20:57.2455766Z .................................................................................................... 7800/9546
2020-01-23T13:21:07.6235934Z .................................................................................................... 7900/9546
2020-01-23T13:21:13.4485561Z ......iiiiiii....................................................................................... 8000/9546
2020-01-23T13:21:27.9582018Z .................................................................................................... 8200/9546
2020-01-23T13:21:39.3065169Z .................................................................................................... 8300/9546
2020-01-23T13:21:51.9568240Z .................................................................................................... 8400/9546
2020-01-23T13:21:58.1739070Z .................................................................................................... 8500/9546
---
2020-01-23T13:24:18.4003262Z  finished in 7.201
2020-01-23T13:24:18.4192920Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-01-23T13:24:18.6250877Z 
2020-01-23T13:24:18.6252412Z running 169 tests
2020-01-23T13:24:21.6524122Z iiii......i........ii..iiii...i....i...........i............i..i..................i....i............ 100/169
2020-01-23T13:24:23.9082253Z i.i.i...iii..iiiiiiiiii.......................iii............ii......
2020-01-23T13:24:23.9084655Z 
2020-01-23T13:24:23.9084855Z  finished in 5.489
2020-01-23T13:24:23.9262267Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-01-23T13:24:24.0965042Z 
---
2020-01-23T13:24:26.0254787Z ---- [codegen-units] codegen-units/partitioning/shared-generics.rs stdout ----
2020-01-23T13:24:26.0254842Z 
2020-01-23T13:24:26.0254882Z These items were contained but should not have been:
2020-01-23T13:24:26.0254907Z 
2020-01-23T13:24:26.0255149Z MONO_ITEM fn core::ptr[0]::drop_in_place[0]<shared_generics_aux::Foo[0]> @@ shared_generics.3a1fbbbh-fallback.cgu[External]
2020-01-23T13:24:26.0255229Z 
2020-01-23T13:24:26.0255538Z thread '[codegen-units] codegen-units/partitioning/shared-generics.rs' panicked at 'explicit panic', src/tools/compiletest/src/runtest.rs:2450:13
2020-01-23T13:24:26.0255609Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2020-01-23T13:24:26.0255638Z 
---
2020-01-23T13:24:26.0261519Z 
2020-01-23T13:24:26.0261735Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:387:22
2020-01-23T13:24:26.0262114Z 
2020-01-23T13:24:26.0262147Z 
2020-01-23T13:24:26.0263511Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/codegen-units" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/codegen-units" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "codegen-units" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-7/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "7.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2020-01-23T13:24:26.0263838Z 
2020-01-23T13:24:26.0263860Z 
2020-01-23T13:24:26.0270334Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2020-01-23T13:24:26.0270560Z Build completed unsuccessfully in 1:01:41
2020-01-23T13:24:26.0270560Z Build completed unsuccessfully in 1:01:41
2020-01-23T13:24:26.0325256Z == clock drift check ==
2020-01-23T13:24:26.0347693Z   local time: Thu Jan 23 13:24:26 UTC 2020
2020-01-23T13:24:26.3279441Z   network time: Thu, 23 Jan 2020 13:24:26 GMT
2020-01-23T13:24:26.3282921Z == end clock drift check ==
2020-01-23T13:24:28.4453876Z 
2020-01-23T13:24:28.4546596Z ##[error]Bash exited with code '1'.
2020-01-23T13:24:28.4560437Z ##[section]Finishing: Run build
2020-01-23T13:24:28.4582305Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/68414/merge to s
2020-01-23T13:24:28.4584176Z Task         : Get sources
2020-01-23T13:24:28.4584213Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-01-23T13:24:28.4584269Z Version      : 1.0.0
2020-01-23T13:24:28.4584301Z Author       : Microsoft
2020-01-23T13:24:28.4584301Z Author       : Microsoft
2020-01-23T13:24:28.4584337Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-01-23T13:24:28.4584395Z ==============================================================================
2020-01-23T13:24:28.9052181Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-01-23T13:24:28.9092980Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/68414/merge to s
2020-01-23T13:24:28.9209353Z Cleaning up task key
2020-01-23T13:24:28.9210255Z Start cleaning up orphan processes.
2020-01-23T13:24:28.9349275Z Terminate orphan process: pid (4390) (python)
2020-01-23T13:24:28.9638090Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-01-23T14:23:23.3185199Z ========================== Starting Command Output ===========================
2020-01-23T14:23:23.3186715Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/389ceeb2-2df0-4602-b6a5-5c0b583a44f8.sh
2020-01-23T14:23:23.3186750Z 
2020-01-23T14:23:23.3189696Z ##[section]Finishing: Disable git automatic line ending conversion
2020-01-23T14:23:23.3196525Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/68414/merge to s
2020-01-23T14:23:23.3198258Z Task         : Get sources
2020-01-23T14:23:23.3198292Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-01-23T14:23:23.3198325Z Version      : 1.0.0
2020-01-23T14:23:23.3198356Z Author       : Microsoft
---
2020-01-23T14:23:25.9491471Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-01-23T14:23:25.9670146Z ##[command]git config gc.auto 0
2020-01-23T14:23:25.9741969Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-01-23T14:23:25.9794229Z ##[command]git config --get-all http.proxy
2020-01-23T14:23:25.9949052Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/68414/merge:refs/remotes/pull/68414/merge

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I don't think I understand fully all the intricacies of what's going on here it all looks reasonable enough to me. Just one thing to double-check below and otherwise r=me

src/librustc/ty/instance.rs Outdated Show resolved Hide resolved
This reduces the amount of invalidated data when new types are
add to upstream crates.
@michaelwoerister
Copy link
Member Author

@bors r=alexcrichton

Thanks for the review!

@bors
Copy link
Contributor

bors commented Jan 23, 2020

📌 Commit 197cc1e has been approved by alexcrichton

@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Jan 23, 2020
@michaelwoerister
Copy link
Member Author

@bors rollup=never

@bors
Copy link
Contributor

bors commented Jan 24, 2020

⌛ Testing commit 197cc1e with merge 73f76b7...

bors added a commit that referenced this pull request Jan 24, 2020
Also share drop-glue when compiling with -Zshare-generics (i.e. at opt-level=0)

This PR adds drop-glue to the set of monomorphizations that can be shared across crates via `-Zshare-generics`.

This version of the PR might have detrimental effects on performance as it makes lots of stuff dependent on a single query results (`upstream_monomorphizations_for(def_id_of_drop_in_place)`). That should be fixable but let's do a perf run first.

Potentially fixes issue #64140. (cc @alexcrichton)
The changes here are related to @matthewjasper's #67332 but should be mostly orthogonal.

r? @ghost
@bors
Copy link
Contributor

bors commented Jan 24, 2020

☀️ Test successful - checks-azure
Approved by: alexcrichton
Pushing 73f76b7 to master...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants