Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustPlatform.cargoSetupHook: fix platform check #260068

Merged
merged 1 commit into from
Oct 11, 2023

Conversation

alyssais
Copy link
Member

@alyssais alyssais commented Oct 9, 2023

Description of changes

Cargo will never need to link for the target platform — that'd be for the package being built to do at runtime. Cargo should know about the build and host linkers.

This fixes e.g. pkgsCross.musl64.fd from x86_64-linux.

Fixes: 67a4f82 ("rust: hooks: fix cross compilation")

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Cargo will never need to link for the target platform — that'd be for
the package being built to do at runtime.  Cargo should know about the
build and host linkers.

This fixes e.g. pkgsCross.musl64.fd from x86_64-linux.

Fixes: 67a4f82 ("rust: hooks: fix cross compilation")
@alyssais alyssais added 6.topic: rust 6.topic: cross-compilation Building packages on a different sort platform than than they will be run on labels Oct 9, 2023
@alyssais alyssais requested review from a user and yu-re-ka October 9, 2023 20:32
@yu-re-ka yu-re-ka merged commit 7262026 into NixOS:staging-next Oct 11, 2023
22 checks passed
@alyssais alyssais deleted the cargoSetupHook-cross branch October 11, 2023 21:38
@ghost
Copy link

ghost commented Oct 22, 2023

Cargo will never need to link for the target platform — that'd be for the package being built to do at runtime.

This is a setup hook, which means it goes in nativeBuildInputs, so stdenv.targetPlatform actually refers to the hostPlatform of the package in which the hook appears.

@alyssais
Copy link
Member Author

Ah, so we should undo this, and replace [target."${rust.toRustTarget stdenv.buildPlatform}"] above with [target."${rust.toRustTarget stdenv.hostPlatform}"], then? Because cargo should definitely never need to know the platform the setup hook was built on.

@ghost
Copy link

ghost commented Oct 23, 2023

Hang on, I'm getting my builds caught up to staging. I'll sort it out.

@yu-re-ka
Copy link
Contributor

For any change, please also investigate if pkgsCross.musl64.fd, pkgsStatic.fd etc. work and produce the right (statically or dynamically linked) binary

@ghost
Copy link

ghost commented Oct 23, 2023

and produce the right (statically or dynamically linked) binary

How do I know if it's the right one? Is there some invocation which was segfaulting before? Or producing the wrong output?

If there is some property of these builds that is important to you, please consider adding a test to tests.cross.sanity. It's the closest thing we have to CI for cross compilation, because OfBorg.

@ghost
Copy link

ghost commented Oct 23, 2023

Ah, so we should undo this

Yes.

and replace [target."${rust.toRustTarget stdenv.buildPlatform}"] above with [target."${rust.toRustTarget stdenv.hostPlatform}"], then? Because cargo should definitely never need to know the platform the setup hook was built on.

I think so. Testing this.

@ghost
Copy link

ghost commented Oct 23, 2023

What was the bug that this PR fixes?

@ghost
Copy link

ghost commented Oct 23, 2023

This is a setup hook, which means it goes in nativeBuildInputs, so stdenv.targetPlatform actually refers to the hostPlatform of the package in which the hook appears.

@Artturin: the dreaded "hooks are not spliced" problem has struck again.

@alyssais
Copy link
Member Author

What was the bug that this PR fixes?

This fixes e.g. pkgsCross.musl64.fd from x86_64-linux.

@Artturin
Copy link
Member

This is a setup hook, which means it goes in nativeBuildInputs, so stdenv.targetPlatform actually refers to the hostPlatform of the package in which the hook appears.

@Artturin: the dreaded "hooks are not spliced" problem has struck again.

That's only a issue inside the python set with its own hooks

pkgsCross.aarch64-multiplatform.__splicedPackages.rustPlatform.cargoSetupHook ? __spliced
true

@ghost
Copy link

ghost commented Oct 23, 2023

Ah, I was thrown off by the title and:

Fixes: 67a4f82 ("rust: hooks: fix cross compilation")

which is a commit not a bug (?)

@ghost
Copy link

ghost commented Oct 23, 2023

That's only a issue inside the python set with its own hooks

They're spliced if you access the hooks directly, but the hook that's finding its way into nativeBuildInputs is not spliced:

diff --git a/pkgs/build-support/rust/build-rust-package/default.nix b/pkgs/build-support/rust/build-rust-package/default.nix
index b75d429ef598..e2f8ad40f23f 100644
--- a/pkgs/build-support/rust/build-rust-package/default.nix
+++ b/pkgs/build-support/rust/build-rust-package/default.nix
@@ -114,7 +114,9 @@ stdenv.mkDerivation ((removeAttrs args [ "depsExtraArgs" "cargoUpdateHook" "carg

   patchRegistryDeps = ./patch-registry-deps;

-  nativeBuildInputs = nativeBuildInputs ++ lib.optionals auditable [
+  nativeBuildInputs =
+    assert args.pname == "fd" && !(cargoSetupHook?__spliced) -> throw "it is not spliced";
+    nativeBuildInputs ++ lib.optionals auditable [
     (buildPackages.cargo-auditable-cargo-wrapper.override {
       inherit cargo cargo-auditable;
     })
$ nix-instantiate . -A pkgsMusl.__splicedPackages.fd
error: it is not spliced

makeRustPlatform is a scope, but it isn't using makeScopeWithSplicing.

@ghost
Copy link

ghost commented Oct 23, 2023

Yeah.

So this PR is correct at the moment, because our rust hooks aren't spliced.

When we get the rust hooks spliced we will need to revert this, however. But until then it's the right change.

#263019

@ghost
Copy link

ghost commented Oct 24, 2023

Proof that the rust hooks are not spliced: #263082 (comment)

uninsane added a commit to uninsane/nixpkgs that referenced this pull request Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: cross-compilation Building packages on a different sort platform than than they will be run on 6.topic: rust 10.rebuild-darwin: 0 10.rebuild-linux: 0 12.approvals: 1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants