Fix Issue 22861 - build the compiler with PGO #13791
Conversation
|
Thanks for your pull request, @maxhaton! Bugzilla references
Testing this PR locallyIf you don't have a local development environment setup, you can use Digger to test this PR: dub run digger -- build "master + dmd#13791" |
| const scope cmd = [ testRunner.targets[0], "compilable", "-j" ~ jobs.to!string ]; | ||
| log("%-(%s %)", cmd); | ||
| if (spawnProcess(cmd, null, Config.init, testDir).wait()) | ||
| stderr.writeln("dmd tests failed! This will not end the PGO build because some data may have been gathered"); |
There was a problem hiding this comment.
Test failures are allowed because of missing tools?
There was a problem hiding this comment.
Which missing tools? If something has gone catastrophically wrong then I guess the build should stop but frankly I view that as unlikely versus most tests passing but some failing due to CI flakyness
There was a problem hiding this comment.
Which missing tools?
Whatever tools are used by postscripts, ... (bash, gdb, ...)
If something has gone catastrophically wrong then I guess the build should stop but frankly I view that as unlikely versus most tests passing but some failing due to CI flakyness
DMD's test usually aren't flaky (contrary to druntime / phobos).
| .deps([buildInstrumentedDmd, testRunner]) | ||
| .commandFunction({ | ||
| // Run dmd test suite to get data | ||
| const scope cmd = [ testRunner.targets[0], "compilable", "-j" ~ jobs.to!string ]; |
There was a problem hiding this comment.
Why restrict this to compilable tests?
There was a problem hiding this comment.
So it runs in less than 20 years. Why would you run the tests?
There was a problem hiding this comment.
More balanced coverage data, especially for the backend (quite a few compilable tests use -o-). Running runnable compilable (and maybe fail_compilation) is fast when omitting the useless permutations for -O -inline ... (disabled by passing ARGS= to run.d)
There was a problem hiding this comment.
Doesn't seem to make any difference in my testing so far.
src/build.d
Outdated
| alias finalDataMerge = methodInit!(BuildRule, (rundBuilder, rundRule) => rundBuilder | ||
| .msg("PGO data has been merged") | ||
| .sources([ testDir.buildPath( "run.d") ]) | ||
| .deps([/* genData */ genData]) |
There was a problem hiding this comment.
| .deps([/* genData */ genData]) | |
| .deps([ genData, genPhobosData ]) |
| @@ -96,6 +98,7 @@ Examples | |||
| ./build.d unittest # runs internal unittests | |||
| ./build.d clean # remove all generated files | |||
| ./build.d generated/linux/release/64/dmd.conf | |||
| ./build.d dmd-pgo # builds dmd with PGO data, currently only LDC is supported | |||
There was a problem hiding this comment.
Why isn't this called release-pgo or pgo-release? Ahh, because it was previously called dmd.
maxhaton
force-pushed
the
amongus
branch
3 times, most recently
from
bce471c
to
5334dd5
Compare
This introduces a new target to build.d, dmd-pgo, which will build a dmd with PGO instrumentation, then run either the phobos or dmd testsuite (both are present in the source code, only the dmd one is set to run at the moment). This data is then merged, at which point the data is used to build a release + LTO build of dmd. This resulting build is significantly faster - on the order of 45% on some programs. Making the release scripts use the PGO build may require further changes.
|
CircleCI passes #13900 🤷 |
|
Superseded by #13900 |
This introduces a new target to build.d, dmd-pgo, which will build a dmd with PGO
instrumentation, then run either the phobos or dmd testsuite (both are present in the
source code, only the dmd one is set to run at the moment). This data is then merged,
at which point the data is used to build a release + LTO build of dmd.
This resulting build is significantly faster - on the order of 45% on some programs.
ENABLE_PGO=1 can be used to make the build script use the dmd-pgo by default, however
something more subtle may be required to make all dmd builds (i.e. releases)
fully pgo-ified