Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derivation platform checking compares against wrong machine #30902

Closed
bgamari opened this issue Oct 28, 2017 · 1 comment
Closed

Derivation platform checking compares against wrong machine #30902

bgamari opened this issue Oct 28, 2017 · 1 comment
Assignees
Labels
6.topic: cross-compilation Building packages on a different platform than they will be used on
Milestone

Comments

@bgamari
Copy link
Contributor

bgamari commented Oct 28, 2017

Issue description

The logic in pkgs/stdenv/generic/check-meta.nix for checking the meta.platforms attribute currently compares meta.platforms against the stdenv.system attribute. In my cross-compilation environment stdenv.system is the double of my build platform, x86_64-linux.

It seems to me like this should rather be compared against stdenv.hostPlatform.system. Unfortunately, this attribute has slightly different semantics than stdenv.system. For instance, in the case of armv7l-hf-multiplatform it is arm-linux whereas the value expected by uboot is something like armv7l-linux.

How much detail should meta.system capture?

Pinging @Ericson2314.

@FRidh FRidh added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label Oct 29, 2017
@Ericson2314 Ericson2314 self-assigned this Nov 8, 2017
@Ericson2314
Copy link
Member

Ericson2314 commented Nov 8, 2017

(tl;dr: 3 is the most important one re fixing the immediate issue.)

  1. It's a Nix rule that the system field of each derivation is the build platform, as a Nix-style 2-component string. No changing that just yet.

  2. stdenv.system can't go away so long is stdenv is a derivation, but I'd like stdenv to be a plain attribute set. [See https://github.com/treewide: Introduce stdenv.cc.bintools #30549#issuecomment-342247583 for other reasons why.]

  3. https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/make-derivation.nix#L171-L173 we can at least make it a bit saner on cross my doing inherit (stdenv.hostPlatform) system; there instead. This will break some cross packages, so I'd like to do after we merge your [RFC] A large batch of cross-compilation fixes #30882 and all the cross tests are fixed (if you fixed cross-built gcc then they will be, yay!).

  4. arm-linux vs rmv7l-linux has to do with me changing everything https://github.com/nixos/nixpkgs/tree/master/lib/systems/ code. armv7l-linux is nicer than arm-linux, but if Nix expects that latter for all 32-bit arm linux native builds (Nix itself won't care for cross), we have a problem and that cross build will have to be adjusted accordingly.

  5. I don't like how meta.platforms works by exhaustive enumeration, when platforms can be tweaked in such fine-grained ways. I suppose the huge space of cross platforms quotiented to just the system field isn't so big, but I still rather do arbitrary predicates or something, as if the space of platforms really was infinite so that was the only way.

@Ericson2314 Ericson2314 mentioned this issue Feb 6, 2018
8 tasks
@Ericson2314 Ericson2314 added this to the 18.03 milestone Feb 11, 2018
Ericson2314 added a commit to obsidiansystems/nixpkgs that referenced this issue Mar 15, 2018
… world

First, we need check against the host platform, not the build platform.
That's simple enough.

Second, we move away from exahustive finite case analysis (i.e.
exhaustively listing all platforms the package builds on). That only
work in a closed-world setting, where we know all platforms we might
build one. But with cross compilation, we may be building for arbitrary
platforms, So we need fancier filters. This is the closed world to open
world change.

The solution is instead of having a list of systems (strings in the form
"foo-bar"), we have a list of of systems or "patterns", i.e. attributes
that partially match the output of the parsers in `lib.systems.parse`.
The "check meta" logic treats the systems strings as an exact whitelist
just as before, but treats the patterns as a fuzzy whitelist,
intersecting the actual `hostPlatform` with the pattern and then
checking for equality. (This is done using `matchAttrs`).

The default convenience lists for `meta.platforms` are now changed to be
lists of patterns (usually a single pattern) in
`lib/systems/for-meta.nix` for maximum flexibility under this new
system.

Fixes NixOS#30902
dotlambda pushed a commit that referenced this issue Mar 20, 2018
… world

First, we need check against the host platform, not the build platform.
That's simple enough.

Second, we move away from exahustive finite case analysis (i.e.
exhaustively listing all platforms the package builds on). That only
work in a closed-world setting, where we know all platforms we might
build one. But with cross compilation, we may be building for arbitrary
platforms, So we need fancier filters. This is the closed world to open
world change.

The solution is instead of having a list of systems (strings in the form
"foo-bar"), we have a list of of systems or "patterns", i.e. attributes
that partially match the output of the parsers in `lib.systems.parse`.
The "check meta" logic treats the systems strings as an exact whitelist
just as before, but treats the patterns as a fuzzy whitelist,
intersecting the actual `hostPlatform` with the pattern and then
checking for equality. (This is done using `matchAttrs`).

The default convenience lists for `meta.platforms` are now changed to be
lists of patterns (usually a single pattern) in
`lib/systems/for-meta.nix` for maximum flexibility under this new
system.

Fixes #30902
Ericson2314 added a commit that referenced this issue Apr 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: cross-compilation Building packages on a different platform than they will be used on
Projects
None yet
Development

No branches or pull requests

3 participants