Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to LLVM 20 #135763

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Update to LLVM 20 #135763

wants to merge 4 commits into from

Conversation

nikic
Copy link
Contributor

@nikic nikic commented Jan 20, 2025

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jan 20, 2025
@nikic
Copy link
Contributor Author

nikic commented Jan 20, 2025

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 20, 2025
@bors
Copy link
Contributor

bors commented Jan 20, 2025

⌛ Trying commit fc35e8c with merge 79efb5f...

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 20, 2025
@bors
Copy link
Contributor

bors commented Jan 20, 2025

☀️ Try build successful - checks-actions
Build commit: 79efb5f (79efb5ff182537ce584a422f953ff24ab8f741db)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (79efb5f): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

mean range count
Regressions ❌
(primary)
5.2% [0.3%, 34.9%] 8
Regressions ❌
(secondary)
0.6% [0.3%, 1.1%] 14
Improvements ✅
(primary)
-1.3% [-4.2%, -0.2%] 250
Improvements ✅
(secondary)
-1.0% [-5.0%, -0.2%] 192
All ❌✅ (primary) -1.1% [-4.2%, 34.9%] 258

Max RSS (memory usage)

Results (primary -1.5%, secondary -1.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.1% [1.1%, 1.2%] 2
Regressions ❌
(secondary)
2.1% [1.1%, 3.6%] 3
Improvements ✅
(primary)
-1.9% [-3.8%, -1.0%] 11
Improvements ✅
(secondary)
-2.1% [-3.4%, -0.9%] 13
All ❌✅ (primary) -1.5% [-3.8%, 1.2%] 13

Cycles

Results (primary -1.8%, secondary -5.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
39.1% [39.1%, 39.1%] 1
Regressions ❌
(secondary)
3.6% [3.6%, 3.6%] 1
Improvements ✅
(primary)
-2.4% [-4.2%, -1.1%] 73
Improvements ✅
(secondary)
-5.2% [-13.6%, -0.9%] 36
All ❌✅ (primary) -1.8% [-4.2%, 39.1%] 74

Binary size

Results (primary -1.2%, secondary -3.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.2% [0.1%, 0.5%] 12
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-1.4% [-7.1%, -0.0%] 96
Improvements ✅
(secondary)
-3.0% [-7.2%, -0.1%] 90
All ❌✅ (primary) -1.2% [-7.1%, 0.5%] 108

Bootstrap: 765.63s -> 749.726s (-2.08%)
Artifact size: 326.02 MiB -> 335.29 MiB (2.85%)

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jan 20, 2025
@rust-log-analyzer

This comment has been minimized.

@nikic
Copy link
Contributor Author

nikic commented Jan 27, 2025

The compiler-rt build (via compiler-builtins) for aarch64-unknown-uefi on test-various fails with:

warning: compiler_builtins@0.1.143: In file included from /checkout/src/llvm-project/compiler-rt/lib/builtins/cpu_model/aarch64.c:52:
warning: compiler_builtins@0.1.143: /checkout/src/llvm-project/compiler-rt/lib/builtins/cpu_model/aarch64/lse_atomics/windows.inc:2:10: fatal error: 'windows.h' file not found                                                                 
warning: compiler_builtins@0.1.143: #include <windows.h>
warning: compiler_builtins@0.1.143:          ^~~~~~~~~~~
warning: compiler_builtins@0.1.143: 1 error generated.
error: failed to run custom build command for `compiler_builtins v0.1.143`   

The compilation command is:

  error occurred: Command "sccache" "clang-11" "-O3" "-ffunction-sections" "-fdata-sections" "--target=aarch64-unknown-windows-gnu" "-I" "/checkout/src/llvm-project/compiler-rt/lib/builtins" "-ffunction-sections" "-fdata-sections" "--target=aarch64-unknown-windows-gnu" "-fno-builtin" "-fvisibility=hidden" "-ffreestanding" "-I/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/aarch64-unknown-uefi/release/build/compiler_builtins-de67acc96632c56e/out/include" "-fomit-frame-pointer" "-ffile-prefix-map=/checkout/src/llvm-project/compiler-rt=." "-DCOMPILER_RT_HAS_FLOAT16" "-DVISIBILITY_HIDDEN" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/aarch64-unknown-uefi/release/build/compiler_builtins-de67acc96632c56e/out/670c1ed24249df13-aarch64.o" "-c" "/checkout/src/llvm-project/compiler-rt/lib/builtins/cpu_model/aarch64.c" with args clang-11 did not execute successfully (status code exit status: 1).

Related upstream change: llvm/llvm-project#116706

@klensy
Copy link
Contributor

klensy commented Jan 27, 2025

clang-11

What's wrong with version?

@nikic
Copy link
Contributor Author

nikic commented Jan 30, 2025

Testing host-aarch64.

@bors try

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 30, 2025
Update to LLVM 20

LLVM 20 GA is scheduled for March 11th. Rust 1.86 will be stable on April 3rd.

* [x] rust-lang#135764
* [x] rust-lang#136134
* [x] rust-lang/compiler-builtins#752
* [ ] Update compiler-builtins

Tested: arm-android, test-various, x86_64-rust-for-linux, x86_64-fuchsia, dist-various-1, dist-various-2, dist-s390x-linux, dist-riscv64-linux, dist-loongarch64-musl

try-job: aarch64-gnu
try-job: aarch64-gnu-debug
try-job: dist-aarch64-linux
@bors
Copy link
Contributor

bors commented Jan 30, 2025

⌛ Trying commit 881aa2d with merge 93cf5c1...

@bors
Copy link
Contributor

bors commented Jan 30, 2025

☀️ Try build successful - checks-actions
Build commit: 93cf5c1 (93cf5c1ddb32e8f3a9fcdd8cd2cf78079ed6c78a)

@nikic
Copy link
Contributor Author

nikic commented Jan 30, 2025

Testing macos.

@bors try

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 30, 2025
Update to LLVM 20

LLVM 20 GA is scheduled for March 11th. Rust 1.86 will be stable on April 3rd.

* [x] rust-lang#135764
* [x] rust-lang#136134
* [x] rust-lang/compiler-builtins#752
* [ ] Update compiler-builtins

Tested: arm-android, test-various, x86_64-rust-for-linux, x86_64-fuchsia, dist-various-1, dist-various-2, dist-s390x-linux, dist-riscv64-linux, dist-loongarch64-musl

try-job: dist-x86_64-apple
try-job: dist-apple-various
try-job: x86_64-apple-1
try-job: x86_64-apple-2
try-job: dist-aarch64-apple
try-job: aarch64-apple
@bors
Copy link
Contributor

bors commented Jan 30, 2025

⌛ Trying commit 881aa2d with merge 92ffaa8...

@bors
Copy link
Contributor

bors commented Jan 30, 2025

☀️ Try build successful - checks-actions
Build commit: 92ffaa8 (92ffaa8cf69eca51dee2346f614ba8e990f37f34)

@nikic
Copy link
Contributor Author

nikic commented Jan 30, 2025

Testing mingw.

@bors try

@bors
Copy link
Contributor

bors commented Jan 30, 2025

⌛ Trying commit 881aa2d with merge 6ff8ac7...

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 30, 2025
Update to LLVM 20

LLVM 20 GA is scheduled for March 11th. Rust 1.86 will be stable on April 3rd.

* [x] rust-lang#135764
* [x] rust-lang#136134
* [x] rust-lang/compiler-builtins#752
* [ ] Update compiler-builtins

Tested: arm-android, test-various, x86_64-rust-for-linux, x86_64-fuchsia, dist-various-1, dist-various-2, dist-s390x-linux, dist-riscv64-linux, dist-loongarch64-musl

try-job: i686-mingw
try-job: x86_64-mingw-1
try-job: x86_64-mingw-2
try-job: dist-i686-mingw
try-job: dist-x86_64-mingw
@nikic
Copy link
Contributor Author

nikic commented Jan 31, 2025

I don't think windows-2025 is going to make a difference here. I'm seeing max-rss of ~2G on that TableGen invocation locally, so it does seem plausible that this could go OOM on i686, especially if parallelism is involved.

Testing a patch to cut memory usage down a bit, though it's only by about 100MB...

@bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 31, 2025
@marcoieni
Copy link
Member

oh right, because i686 can only use a maximum of 4GB RAM. 💡
More reasons to demote this target maybe? 😅 I'll leave it to you to figure out 👍

@nikic
Copy link
Contributor Author

nikic commented Jan 31, 2025

@bors try

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 31, 2025
Update to LLVM 20

LLVM 20 GA is scheduled for March 11th. Rust 1.86 will be stable on April 3rd.

* [x] rust-lang#135764
* [x] rust-lang#136134
* [x] rust-lang/compiler-builtins#752
* [ ] Update compiler-builtins

Tested: host-x86_64, host-aarch64

try-job: i686-mingw
try-job: x86_64-mingw-1
try-job: x86_64-mingw-2
try-job: dist-i686-mingw
try-job: dist-x86_64-mingw
@bors
Copy link
Contributor

bors commented Jan 31, 2025

⌛ Trying commit 9539a55 with merge a093dde...

@mati865
Copy link
Contributor

mati865 commented Jan 31, 2025

oh right, because i686 can only use a maximum of 4GB RAM. 💡
More reasons to demote this target maybe? 😅 I'll leave it to you to figure out 👍

Actually, it's half that size unless the program is linked with a special flag (the codebase must also support this). So, with 2 GiB Nikita has observed, we are right on the edge.

The same issue can manifest on any 32-bit Windows system (including those using MSVC). Note that it is LLVM's tblgen that fails, not GCC or ld.bfd.

LLVM generally does not cope well with 32-bit hosts, but so far, the problem has mostly been due to mmap having insufficient address space.

One solution is cross-compiling with a 64-bit toolchain, but that requires changes to the entire build process.

@bors
Copy link
Contributor

bors commented Jan 31, 2025

☀️ Try build successful - checks-actions
Build commit: a093dde (a093dde8bafff0c2c30e443df480946ec24196fe)

@nikic
Copy link
Contributor Author

nikic commented Jan 31, 2025

Looks like that was enough. Now testing msvc...

@bors try

@bors
Copy link
Contributor

bors commented Jan 31, 2025

⌛ Trying commit 9539a55 with merge 23c742f...

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 31, 2025
Update to LLVM 20

LLVM 20 GA is scheduled for March 11th. Rust 1.86 will be stable on April 3rd.

* [x] rust-lang#135764
* [x] rust-lang#136134
* [x] rust-lang/compiler-builtins#752
* [ ] llvm/llvm-project#125287
* [ ] Update compiler-builtins

Tested: host-x86_64, host-aarch64, mingw

try-job: x86_64-msvc-1
try-job: x86_64-msvc-2
try-job: i686-msvc-1
try-job: i686-msvc-2
try-job: dist-x86_64-msvc
try-job: dist-i686-msvc
try-job: dist-aarch64-msvc
@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Feb 1, 2025

💔 Test failed - checks-actions

@bors bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Feb 1, 2025
@nikic
Copy link
Contributor Author

nikic commented Feb 1, 2025

Looks like the embedded command line arguments in https://github.com/rust-lang/rust/blob/master/tests/run-make/pdb-buildinfo-cl-cmd/filecheck.txt are no longer quoted. I think this is because llvm/llvm-project@e190d07 moved the command line printing to the frontend, but the Rust adjustment in ad0eceb just prints the arguments directly, instead of going through llvm::sys::printArg.

@matthiaskrgr
Copy link
Member

matthiaskrgr commented Feb 1, 2025

@bors r-

note that bors retry means "please try to merge again (as in, after approval, CI failed spuriously) and ´bory try´ means "please run a try job" so "retry" is not a "please re-try the failed try" job! 😅

@bors bors removed the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Feb 1, 2025
@the8472
Copy link
Member

the8472 commented Feb 3, 2025

Actually, it's half that size unless the program is linked with a special flag (the codebase must also support this). So, with 2 GiB Nikita has observed, we are right on the edge.

The flag can also be set after linking via editbin /LARGEADDRESSAWARE <executable>. Might be worth a try, programs doing pointer shenanigans aren't that common.

The formatting of the command line arguments has been moved to the
frontend in:
llvm/llvm-project@e190d07

However, the Rust logic introduced in
rust-lang@ad0eceb
did not replicate the previous argument quoting behavior.
@nikic
Copy link
Contributor Author

nikic commented Feb 3, 2025

@bors try

bors added a commit to rust-lang-ci/rust that referenced this pull request Feb 3, 2025
Update to LLVM 20

LLVM 20 GA is scheduled for March 11th. Rust 1.86 will be stable on April 3rd.

* [x] rust-lang#135764
* [x] rust-lang#136134
* [x] rust-lang/compiler-builtins#752
* [ ] llvm/llvm-project#125287
* [ ] Update compiler-builtins (blocked on rust-lang#135501)

Tested: host-x86_64, host-aarch64, apple, mingw

try-job: x86_64-msvc-1
try-job: i686-msvc-1
try-job: dist-aarch64-msvc
try-job: x86_64-msvc-ext1
try-job: x86_64-msvc-ext2
try-job: x86_64-msvc-ext3
@bors
Copy link
Contributor

bors commented Feb 3, 2025

⌛ Trying commit ee8284e with merge ad23dfe...

@mati865
Copy link
Contributor

mati865 commented Feb 3, 2025

The flag can also be set after linking via editbin /LARGEADDRESSAWARE <executable>. Might be worth a try, programs doing pointer shenanigans aren't that common.

Editbin is not the most manageable approach because you would have to modify LLVM's build system (notice it failed halfway through the build), but you can append linker flags to all Windows platforms.

I am not familiar enough with LLVM code to judge whether something would break, but yes, that might be another option to try.

@bors
Copy link
Contributor

bors commented Feb 3, 2025

☀️ Try build successful - checks-actions
Build commit: ad23dfe (ad23dfe0eb9153bd71988323086e143f7b3947f6)

@tgross35
Copy link
Contributor

tgross35 commented Feb 6, 2025

* [ ]  Update compiler-builtins (blocked on [Inject `compiler_builtins` during postprocessing and ensure it is made private #135501](https://github.com/rust-lang/rust/pull/135501))

I wound up just doing the revert to unblock this, should be done as of #136537

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
perf-regression Performance regression. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.