freshBootstrapTools: rename from stdenvBootstrapTools #182058
Conversation
If name collisions cause |
|
Looking at https://hydra.nixos.org/jobset/nixpkgs/trunk#tabs-configuration |
|
@ofborg eval |
|
After the change we can see both present in top level: Not sure why UPDATE: possibly intended: all attrsets are scrubbed down to empty (including |
`stdenvBootstrapTools` was defined in both `nixpkgs` and `releases`
jobsets. This makes hydra view a bit confusing:
$ git grep -F 'stdenvBootstrapTools =' | cat
pkgs/top-level/all-packages.nix: stdenvBootstrapTools = if stdenv.hostPlatform.isDarwin then
pkgs/top-level/release.nix: stdenvBootstrapTools = with lib;
The change renames jobset to be distinct. At least it improves grep
experience.
trofi
force-pushed
the
rename-stdenvBootstrapTools-to-freshBootstrapTools
branch
from
42be626
to
94f95bf
Compare
|
Looks like plain rename does not work anymore and fails eval. I wonder if 5643714 somehow relies on new meaning of stdenvBootstrapTools. I don't see how exacly how they are related. Maybe @alyssais knows? I suspect things were broken for a while. Before this change: After the change: |
|
I suspect we need to adapt below to avoid removed stdenvBootstrapTools = with lib;
genAttrs systemsWithAnySupport
(system: {
inherit
(import ../stdenv/linux/make-bootstrap-tools.nix {
localSystem = { inherit system; };
})
dist test;
}) |
This reverts commit 5643714. The change was based on e663518 ("all-packages.nix: add bootstrapTools to top-level.nix"). Unfortunately that change had intended side-effect in releases.nix that turned stdenvBootstrapTools into a no-op. As a result 5643714 "stdenvBootstrapTools: inherit {cross,local}System" change dropped the locaSystem argument without evaluation failure. Once the accidental change was fixed unexpected `localSystem` parameter started failing evals: $ nix build -f pkgs/top-level/release.nix stdenvBootstrapTools.x86_64-linux.dist error: anonymous function at pkgs/stdenv/linux/make-bootstrap-tools.nix:1:1 called with unexpected argument 'localSystem' at /home/slyfox/dev/git/nixpkgs-master/pkgs/top-level/release.nix:169:16: 168| inherit 169| (import ../stdenv/linux/make-bootstrap-tools.nix { | ^ 170| localSystem = { inherit system; }; Let's roll it back cleanly to restore stdenvBootstrapTools builds on hydra.
Added a rollback CL to this PR (don't know how to fix it forward). Rollback restores |
|
I have a stupid question: are the configuration files for the Hydra instance running on I think I've run into this problem before. |
|
[caveat: I never created hydra jobsets] I think the job definition is clicked on hydra UI directly: you pick a repository, branch, expression and possibly a set of platforms to run on. https://hydra.nixos.org/jobset/nixpkgs/trunk#tabs-configuration |
|
you pick a repository, branch, expression and possibly a set of platforms to run on Yes, but where is the list of repository/branch/expressions that are run automatically? In other words, from your link:
Where did pkgs/top-level/release.nix come from? Similarly, on this page, where did pkgs/top-level/release-cross.nix come from? There must be a list of "entry points into nixpkgs" somewhere in the hydra.nixos.org configuration. I've never been able to find that list. I suspect that if the root cause of the problem in this PR is some kind of name-collision, it would be in whatever merges these entry points together. |
Gah! My apologies! I hope I did not butcher it while restoring it. No idea why I cleackied
I think people typed it in using web UI. I would guess administrator privileges give you that. json API exposes it as well: https://github.com/NixOS/hydra/blob/master/hydra-api.yaml#L148 |
I think all of merging happens right within I guess |
|
Personally I don't care what the package is called. But I think the bug here isn't the package name, it's the fact that name collisions cause jobs to be dropped silently. Simply renaming the package is a band-aid; this will happen again somewhere else. The way it got this name in the first place is out of a desire to have the same name for jobs in both |
Is your suggestion to not rename the package and make it work? Or have an eval to fail loudly? (that would still require a rename). Failing loudly should be easier by slightly changing the way attrs are merged. Say, some equivalent of |
Having it fail loudly and then changing the name to unfail it sounds great. I'm still astonished that Hydra doesn't send notifications of any kind and people seem to be okay with that. That is pretty demotivating. |
…local jobs
This change exposes symbol override that accidentally caused job loss on hydra:
$ nix repl ./release.nix
error: jobs: Unexpected attribute collision between 'jobs' and 'pkgs': stdenvBootstrapTools
Added assert makes sure attribute clashes would not be re-introduced.
Added a
That's probably worth discussing with infrastrusture team. Maybe in NixOS/infra#51? I personally would prefer an RSS feed for package state changes I care about. I don't trust email delivery. |
|
--- a/pkgs/top-level/release.nix
+++ b/pkgs/top-level/release.nix
@@ -31,6 +31,7 @@ let
] (arch: builtins.elem "${arch}-darwin" systemsWithAnySupport);
jobs =
+ let nonPackageJobs =
{ tarball = import ./make-tarball.nix { inherit pkgs nixpkgs officialRelease supportedSystems; };
metrics = import ./metrics.nix { inherit pkgs nixpkgs; };
@@ -193,7 +194,19 @@ let
};
};
- } // (mapTestOn ((packagePlatforms pkgs) // {
+ };
+ # Do not allow attribute collision between jobs inserted in
+ # 'nonPackageAttrs' and jobs pulled in from 'pkgs'.
+ # Conflicts usually cause silent job drops like in
+ # https://github.com/NixOS/nixpkgs/pull/182058
+ nonPackageAttrs = lib.attrNames nonPackageJobs;
+ packageAttrs = lib.attrNames pkgs;
+ attributeCollisions = lib.intersectLists nonPackageAttrs packageAttrs;
+ in assert lib.assertMsg
+ (attributeCollisions == [])
+ "jobs: Unexpected attribute collision between 'jobs' and 'pkgs': ${lib.concatStringsSep " " attributeCollisions}";
+
+ nonPackageJobs // (mapTestOn ((packagePlatforms pkgs) // {
haskell.compiler = packagePlatforms pkgs.haskell.compiler;
haskellPackages = packagePlatforms pkgs.haskellPackages;
idrisPackages = packagePlatforms pkgs.idrisPackages; |
| (attributeCollisions == []) | ||
| "jobs: Unexpected attribute collision between 'jobs' and 'pkgs': ${lib.concatStringsSep " " attributeCollisions}"; | ||
|
|
||
| nonPackageJobs // (mapTestOn ((packagePlatforms pkgs) // { |
There was a problem hiding this comment.
If somebody comes along later and adds a third set to this update (a//b) nothing will warn them that they forgot to also add it to the separate check above (see previous comment).
Let's factor out the "union these attrsets while checking that they are disjoint" since it is something nixpkgs ought to use more often. Please consider cherry-picking 8b3eea55b66381a1bf3a3f0baa0ef48a5ed786f9
| # Distribution only for now | ||
| inherit (bootstrap) dist; | ||
| }; | ||
| let nonPackageJobs = |
There was a problem hiding this comment.
The multiply-nested lets here are getting hard to read, and not really necessary... please consider cherry-picking 2dd9e12478a3ed69a80ecc986696c7cc6059d4b1
Any kind of notification would be great. Right now we have zero kinds. |
|
I think this PR gets only more bloated code-wise. I'm declaring the defeat. I hope the failure mode is clear enough for others to fix it properly. |
ghost
mentioned this pull request
Whenever I re-indent code I haven't touched, git always makes me regret it... |
stdenvBootstrapToolswas defined in bothnixpkgsandreleasesjobsets. This makes hydra view a bit confusing:
The change renames jobset to be distinct. At least it improves grep
experience.
Description of changes
Things done
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)nixos/doc/manual/md-to-db.shto update generated release notes