Ensure all jobset attributes evaluate, and check that in CT #269356
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
6.topic: nixos
6.topic: GNOME
ghost
changed the title
ghost
mentioned this pull request
MeasurementsAccording to GNU Time Compared to |
ghost
marked this pull request as ready for review
ghost
requested review from
piegamesde,
jtojnar,
adisbladis,
ttuegel,
edwtjo,
Mic92,
zowoq,
winterqt,
figsoda,
stigtsp,
zakame and
dasJ
as code owners
added 2 commits
December 15, 2023 04:21
…ckages, most of them are broken Since Hydra does not build unfree packages an astonishing proportion of them are broken yet not marked meta.broken.
|
Rebased. |
added 6 commits
December 15, 2023 05:13
This file walks the entire nixpkgs tree and emits a superset of all
release attrnames in only 44 seconds on a 3ghz CPU, using 5 gbytes
of memory. By comparison, on the same CPU the `nix-env` hack used
by ofborg on every PR submission requires 41 *minutes* and peaks at
60 gbytes, even with checkMeta turned off. Full details below.
This is:
- 46x faster (or 2.1% of the elapsed time)
- 12.5x less memory (or 8.0% of the peak memory usage)
In order to replace the ofborg check, this list of attrnames must
then be post-filtered for platform-relevance. However, crucially,
the post-filtering can be done *in parallel* on multiple cores by
splitting the attrname list in to chunks. Generating the list of
attrnames cannot be parallelized because it is a single-threaded
cppnix task.
This PR also adds `recurseForDerivations` where necessary within
nixpkgs in order to make this possible -- it screens out various
non-tryEval-catchable failures and infinite recursions. Before
undraftifying, I will add an invocation of this command to the CI
tests, to ensure that the work performed here is not immediately
undone. My next PR will then add an additional CI check confirming
that the emitted attrpaths are in fact a superset of the release
attrpaths calculated by the slow-memory-hog ofborg method.
I have manually confirmed that this is the case at the tip commit of
this PR, but we need CI to make sure this remains true until ofborg
switches to this more-efficient method of calculation; at that point
the superset-check can be dropped.
According to GNU Time,
Command being timed: "nix-instantiate --eval --strict --json pkgs/top-level/release-attrpaths-superset.nix -A names"
User time (seconds): 44.88
System time (seconds): 8.09
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:53.20
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 4823028
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 3611240
Voluntary context switches: 113
Involuntary context switches: 949
Swaps: 0
File system inputs: 1480
File system outputs: 5944
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Compared to release-outpaths.nix:
Command being timed: "nix-env -qaP --no-name --out-path --arg checkMeta false --argstr path /git/work/pr/release-outpaths -f pkgs/top-level/release-outpaths.nix"
User time (seconds): 2120.67
System time (seconds): 337.80
Percent of CPU this job got: 98%
Elapsed (wall clock) time (h:mm:ss or m:ss): 41:37.91
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 60171768
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 2
Minor (reclaiming a frame) page faults: 230608113
Voluntary context switches: 8876
Involuntary context switches: 22275
Swaps: 0
File system inputs: 62624
File system outputs: 72
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
This derivation verifies that all jobset attributes can be evaluated under tryEval without producing any non-catchable errors or causing infinite recursion.
This commit temporarily adds pkgs/test/release to the lib/tests/release.nix test suite, because ofborg already knows about that entry point. We should move the list of test entry points out of ofborg and into a central place in nixpkgs: #272591 Once we do that we won't need to have this ugly kludge in an inappropriate place.
…unsupported" We have packages that use `meta.platforms = []` as a sort of synonym for `broken = true`. Without this commit, the attrnames for those jobs will end up in the list of attrnames which are expected to build, even though they are not expected to build.
infinisil
approved these changes
Dec 15, 2023
This was referenced Jul 1, 2024
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
aka "eval checks go brrrr"
Please review the commits individually. Produced as part of the non-Hydra tooling used to validate
Description of changes
This PR adds
release-attrpaths-superset.nix, which walks the entire nixpkgs tree and emits a superset of all release attrnames in only 44 seconds on a 3ghz CPU, using less than 5 gbytes of memory. By comparison, on the same CPU thenix-env -qaPhack used by ofborg on every PR submission requires waiting 25 minutes and peaks at 58 gbytes, even with checkMeta turned off. Full details below. This is:If all you need is the attrnames -- for example to launch a mass rebuild -- then you're done. If you want to replace the ofborg
outpaths.nixstep, see the follow-up PR:Validation
To confirm that the
release-attrpaths-superset.nixreally are a superset of the slower method, runAdditional CT (Continuous Testing) check
This PR adds
recurseForRelease=falsewhere necessary within nixpkgs in order to make this possible -- it screens out infinite recursions and several non-tryEval-catchable failures (for example, passing a mistyped argument to a builtin, which is an uncatchable error) and infinite recursions.Finding all the places where it was necessary to add this was a lot of tedious work, so to make sure that work is not undone,
lib/tests/release.nixnow verifies thatrelease-attrpaths-superset.nixevaluates successfully.Although it doesn't really fit in
lib/tests, there isn't anywhere else it can be placed and still be run by ofborg on every commit. If some other such place materializes in the future I would be happy to relocate it, but since ofborg is out-of-tree this PR should not block on that happening.Fixes