CI: update `rustc-perf` version used in CI and also the corresponding PGO benchmarks #95724

Kobzol · 2022-04-06T07:28:51Z

The old version was from May 2021. The rustc-perf benchmarks have seen a significant overhaul recently, so let's see if the new benchmarks can improve PGO performance.

rust-highfive · 2022-04-06T07:28:54Z

r? @kennytm

(rust-highfive has picked a reviewer for you, use r? to override)

Kobzol · 2022-04-06T07:29:08Z

@bors try @rust-timer queue

rust-timer · 2022-04-06T07:29:10Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2022-04-06T07:29:16Z

⌛ Trying commit 4a31a7ddcea8fe9805c3e5ec109cfa7dd98dc49c with merge 4e588bd7921c625cbd174d36019b756c3f4a89e7...

Kobzol · 2022-04-06T07:45:31Z

@bors try

bors · 2022-04-06T07:45:39Z

⌛ Trying commit 78652850eee351a42819ebb16408e0e23189186d with merge eeb04f75b8aca0f9a59de245b827af8601417357...

bors · 2022-04-06T08:59:14Z

💔 Test failed - checks-actions

Kobzol · 2022-04-06T10:24:01Z

@bors try

bors · 2022-04-06T10:24:08Z

⌛ Trying commit 87b417e9b5009aa34a5a7880c99f79bdae125981 with merge acb059f589ddb9e518c7a0afe86f34b8b5fbff6b...

bors · 2022-04-06T11:52:59Z

☀️ Try build successful - checks-actions
Build commit: acb059f589ddb9e518c7a0afe86f34b8b5fbff6b (acb059f589ddb9e518c7a0afe86f34b8b5fbff6b)

rust-timer · 2022-04-06T11:53:00Z

Queued acb059f589ddb9e518c7a0afe86f34b8b5fbff6b with parent ce1a813, future comparison URL.

rust-timer · 2022-04-06T13:32:23Z

Finished benchmarking commit (acb059f589ddb9e518c7a0afe86f34b8b5fbff6b): comparison url.

Summary:

Primary benchmarks: 🎉 relevant improvements found
Secondary benchmarks: mixed results

	Regressions 😿 (primary)	Regressions 😿 (secondary)	Improvements 🎉 (primary)	Improvements 🎉 (secondary)	All 😿 🎉 (primary)
count¹	1	19	90	96	91
mean²	0.2%	1.1%	-0.7%	-0.8%	-0.7%
max	0.2%	1.8%	-2.1%	-2.3%	-2.1%

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR led to changes in compiler perf.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf -perf-regression

number of relevant changes ↩
the arithmetic mean of the percent change ↩

Kobzol · 2022-04-06T13:35:54Z

Results look rather good, slight upgrades in most benchmarks. Since we now want to optimize for the benchmarks in rustc-perf, I think that this is worth merging.

r? @nnethercote

…enchmarks

nnethercote · 2022-04-07T09:09:12Z

@bors r+

bors · 2022-04-07T09:09:13Z

📌 Commit 49efa23 has been approved by nnethercote

bors · 2022-04-07T18:14:35Z

⌛ Testing commit 49efa23 with merge 051ddcc0cb2111e36d52105f0c3d60dc7a63b5c1...

bors · 2022-04-07T19:37:39Z

💔 Test failed - checks-actions

rust-log-analyzer · 2022-04-07T19:37:48Z

The job dist-apple-various failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

lqd · 2022-04-07T19:53:53Z

@bors retry weird failure on dist-apple-various, no apparent error

retrying once to see if it's a fluke

bors · 2022-04-08T04:12:58Z

⌛ Testing commit 49efa23 with merge 4aa34e17b4eae914cfe2b717b97388515fb8681d...

bors · 2022-04-08T04:55:35Z

💔 Test failed - checks-actions

Kobzol · 2022-04-08T06:38:54Z

That.. looks spurious again?

rust-log-analyzer · 2022-04-08T06:52:31Z

The job i686-msvc-1 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

Some tests failed in compiletest suite=ui-fulldeps mode=ui host=i686-pc-windows-msvc target=i686-pc-windows-msvc

error: test run failed!
status: exit code: 101
command: PATH="D:\a\rust\rust\build\i686-pc-windows-msvc\stage2\lib\rustlib\i686-pc-windows-msvc\lib;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x86;D:\a\rust\rust\build\i686-pc-windows-msvc\stage0-bootstrap-tools\i686-pc-windows-msvc\release\deps;D:\a\rust\rust\build\i686-pc-windows-msvc\stage0\bin;D:\a\rust\rust\ninja;D:\a\rust\rust\msys2\mingw32\bin;C:\hostedtoolcache\windows\Python\3.10.4\x64\Scripts;C:\hostedtoolcache\windows\Python\3.10.4\x64;C:\msys64\usr\bin;D:\a\rust\rust\sccache;C:\Program Files\MongoDB\Server\5.0\bin;C:\aliyun-cli;C:\vcpkg;C:\cf-cli;C:\Program Files (x86)\NSIS;C:\tools\zstd;C:\Program Files\Mercurial;C:\hostedtoolcache\windows\stack\2.7.5\x64;C:\cabal\bin;C:\ghcup\bin;C:\tools\ghc-9.2.2\bin;C:\Program Files\dotnet;C:\mysql\bin;C:\Program Files\R\R-4.1.3\bin\x64;C:\SeleniumWebDrivers\GeckoDriver;C:\Program Files (x86)\sbt\bin;C:\Program Files (x86)\GitHub CLI;C:\Program Files\Git\bin;C:\Program Files (x86)\pipx_bin;C:\hostedtoolcache\windows\go\1.15.15\x64\bin;C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts;C:\hostedtoolcache\windows\Python\3.7.9\x64;C:\hostedtoolcache\windows\Ruby\2.5.9\x64\bin;C:\tools\kotlinc\bin;C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.322-6\x64\bin;C:\npm\prefix;C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;C:\ProgramData\kind;C:\Program Files\Eclipse Foundation\jdk-8.0.302.8-hotspot\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\ProgramData\Chocolatey\bin;C:\Program Files\Docker;C:\Program Files\PowerShell\7;C:\Program Files\Microsoft\Web Platform Installer;C:\Program Files\dotnet;C:\Program Files\Microsoft SQL Server\130\Tools\Binn;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn;C:\Program Files\nodejs;C:\Program Files\OpenSSL\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;C:\Program Files\TortoiseSVN\bin;C:\Program Files\CMake\bin;C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.5\bin;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Program Files\GitHub CLI;C:\tools\php;C:\Program Files (x86)\sbt\bin;C:\SeleniumWebDrivers\ChromeDriver;C:\SeleniumWebDrivers\EdgeDriver;C:\Program Files\Amazon\AWSCLIV2;C:\Program Files\Amazon\SessionManagerPlugin\bin;C:\Program Files\Amazon\AWSSAMCLI\bin;C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin;C:\Program Files (x86)\Microsoft BizTalk Server;C:\Program Files\LLVM\bin;C:\Users\runneradmin\.dotnet\tools;C:\Users\runneradmin\.cargo\bin;C:\Users\runneradmin\AppData\Local\Microsoft\WindowsApps" "D:\\a\\rust\\rust\\build\\i686-pc-windows-msvc\\test\\ui-fulldeps\\stdio-from\\a.exe"
stdout: none
--- stderr -------------------------------
I/O error: operation failed to complete synchronously
thread 'main' panicked at 'assertion failed: child2.wait()?.success()', D:\a\rust\rust\src/test\ui-fulldeps\stdio-from.rs:50:5
------------------------------------------




failures:
    [ui] src/test\ui-fulldeps\stdio-from.rs

test result: FAILED. 65 passed; 1 failed; 1 ignored; 0 measured; 0 filtered out; finished in 26.49s

Build completed unsuccessfully in 0:36:54
make: *** [Makefile:72: ci-subset-1] Error 1

lqd · 2022-04-08T06:58:04Z

@bors retry #95759

bors · 2022-04-09T07:53:37Z

⌛ Testing commit 49efa23 with merge 399dd80...

bors · 2022-04-09T10:07:05Z

☀️ Test successful - checks-actions
Approved by: nnethercote
Pushing 399dd80 to master...

rust-timer · 2022-04-09T11:20:13Z

Finished benchmarking commit (399dd80): comparison url.

Summary:

Primary benchmarks: 🎉 relevant improvements found
Secondary benchmarks: mixed results

	Regressions 😿 (primary)	Regressions 😿 (secondary)	Improvements 🎉 (primary)	Improvements 🎉 (secondary)	All 😿 🎉 (primary)
count¹	0	13	102	101	102
mean²	N/A	1.0%	-0.7%	-1.0%	-0.7%
max	N/A	1.3%	-2.2%	-2.7%	-2.2%

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

number of relevant changes ↩
the arithmetic mean of the percent change ↩

rust-highfive assigned kennytm Apr 6, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 6, 2022

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Apr 6, 2022

This comment has been minimized.

Sign in to view

Kobzol force-pushed the ci-update-rustc-perf branch from 4a31a7d to 7865285 Compare April 6, 2022 07:45

This comment has been minimized.

Sign in to view

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 6, 2022

Kobzol force-pushed the ci-update-rustc-perf branch from 7865285 to 87b417e Compare April 6, 2022 10:23

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Apr 6, 2022

rust-highfive assigned nnethercote and unassigned kennytm Apr 6, 2022

Update rustc-perf version used in CI and also the corresponding PGO b…

49efa23

…enchmarks

Kobzol force-pushed the ci-update-rustc-perf branch from 99bebd1 to 49efa23 Compare April 7, 2022 07:42

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Apr 7, 2022

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 7, 2022

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Apr 8, 2022

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 8, 2022

bors added the merged-by-bors This PR was explicitly merged by bors. label Apr 9, 2022

bors merged commit 399dd80 into rust-lang:master Apr 9, 2022

rustbot added this to the 1.62.0 milestone Apr 9, 2022

Kobzol deleted the ci-update-rustc-perf branch April 9, 2022 10:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: update `rustc-perf` version used in CI and also the corresponding PGO benchmarks #95724

CI: update `rustc-perf` version used in CI and also the corresponding PGO benchmarks #95724

Kobzol commented Apr 6, 2022

rust-highfive commented Apr 6, 2022

Kobzol commented Apr 6, 2022

rust-timer commented Apr 6, 2022

bors commented Apr 6, 2022

This comment has been minimized.

Kobzol commented Apr 6, 2022

bors commented Apr 6, 2022

This comment has been minimized.

This comment has been minimized.

bors commented Apr 6, 2022

Kobzol commented Apr 6, 2022

bors commented Apr 6, 2022

bors commented Apr 6, 2022

rust-timer commented Apr 6, 2022

rust-timer commented Apr 6, 2022

Kobzol commented Apr 6, 2022

nnethercote commented Apr 7, 2022

bors commented Apr 7, 2022

bors commented Apr 7, 2022

bors commented Apr 7, 2022

rust-log-analyzer commented Apr 7, 2022

lqd commented Apr 7, 2022

bors commented Apr 8, 2022

bors commented Apr 8, 2022

Kobzol commented Apr 8, 2022

rust-log-analyzer commented Apr 8, 2022

lqd commented Apr 8, 2022

bors commented Apr 9, 2022

bors commented Apr 9, 2022

rust-timer commented Apr 9, 2022

CI: update rustc-perf version used in CI and also the corresponding PGO benchmarks #95724

CI: update rustc-perf version used in CI and also the corresponding PGO benchmarks #95724

Conversation

Kobzol commented Apr 6, 2022

rust-highfive commented Apr 6, 2022

Kobzol commented Apr 6, 2022

rust-timer commented Apr 6, 2022

bors commented Apr 6, 2022

This comment has been minimized.

Kobzol commented Apr 6, 2022

bors commented Apr 6, 2022

This comment has been minimized.

This comment has been minimized.

bors commented Apr 6, 2022

Kobzol commented Apr 6, 2022

bors commented Apr 6, 2022

bors commented Apr 6, 2022

rust-timer commented Apr 6, 2022

rust-timer commented Apr 6, 2022

Footnotes

Kobzol commented Apr 6, 2022

nnethercote commented Apr 7, 2022

bors commented Apr 7, 2022

bors commented Apr 7, 2022

bors commented Apr 7, 2022

rust-log-analyzer commented Apr 7, 2022

lqd commented Apr 7, 2022

bors commented Apr 8, 2022

bors commented Apr 8, 2022

Kobzol commented Apr 8, 2022

rust-log-analyzer commented Apr 8, 2022

lqd commented Apr 8, 2022

bors commented Apr 9, 2022

bors commented Apr 9, 2022

rust-timer commented Apr 9, 2022

Footnotes

CI: update `rustc-perf` version used in CI and also the corresponding PGO benchmarks #95724

CI: update `rustc-perf` version used in CI and also the corresponding PGO benchmarks #95724