Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

git: Don't depend curl-config when cross-compiling. #61552

Closed
wants to merge 1 commit into from

Conversation

nh2
Copy link
Contributor

@nh2 nh2 commented May 15, 2019

Motivation for this change

Fixes building with pkgsStatic, where otherwise curl linker flags
would not be correctly determined.

Better approach than in #61527.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Fixes building with `pkgsStatic`, where otherwise curl linker flags
would not be correctly determined.
@@ -76,6 +77,9 @@ stdenv.mkDerivation {
configureFlags = stdenv.lib.optionals (stdenv.buildPlatform != stdenv.hostPlatform) [
"ac_cv_fread_reads_directories=yes"
"ac_cv_snprintf_returns_bogus=no"
# Provide curl manually, do not rely on `curl-config`
# See also `CURL_LDFLAGS` below.
"--with-curl=${symlinkJoin { name = "curl-libs-and-headers"; paths = [ curl.out curl.dev ]; }}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually maybe try CURL_CONFIG=${curl.dev}/bin/curl-config? from here: https://github.com/git/git/blob/master/configure.ac#L597

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using symlinkJoin is usually a bad idea

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using symlinkJoin is usually a bad idea

@matthewbauer Why?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It breaks the closure size reductions from multiple outputs. When you link to that lib directory, you will also get the headers in closures.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see. That makes sense.

What should I do though, given that git has only a single flag as opposed to one for headers and libs each?

I can try the curl-config approach, but using explicit flags sems to have less surprises so that might be preferable.

Do you no longer have the concerns about curl-config from #61527, or is the difference of your proposal above that we'd pass the CURL_CONFIG manually and that would somehow be better / address your previous concern of

Otherwise you wind up linking to the nonstatic curl I think

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewbauer A short ping for my questions above :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry. Yeah you can use curl-config, you just can't put it in nativeBuildInputs, otherwise you get the build system's curl-config binary, not the targeted system. I believe putting "CURL_CONFIG=${curl.dev}/bin/curl-config" in configureFlags should be okay (if a little awkward).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewbauer Except that curl-config won't work if you're cross-compiling to a different architecture, e.g. armv7l.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kisik21 Some packages natively compile the config tools that go in the dev output precisely for this reason. It's ugly either way, but doing that is practical.

@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild 10.rebuild-linux: 0 This PR does not cause any packages to rebuild labels May 15, 2019
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/proposal-a-dedicated-cross-compiling-team/3824/2

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/proposal-a-dedicated-cross-compiling-team/3824/1

@vikanezrimaya
Copy link
Member

I tried to build it with the following: (perl support disabled because of #66741)

nix-build -E 'with import <nixpkgs> { crossSystem = { system = "armv7l-linux"; }; system = builtins.currentSystem; }; git.override { perlSupport = false; }'

It got me the following:

/build/.attr-0: line 3: /nix/store/wlr38bsv3n6ipxi27mcln2ajriwfwag3-lndir-1.0.3-armv7l-unknown-linux-gnueabihf/bin/lndir: No such file or directory
builder for '/nix/store/qm9fpy3dc2ky4mc7xqqkdmbsv6c50gj9-curl-libs-and-headers.drv' failed with exit code 127

I think that it's a bug somewhere in the symlinkJoin function.

@matthewbauer matthewbauer added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label Aug 28, 2019
@nh2
Copy link
Contributor Author

nh2 commented Nov 3, 2019

I wanted to look more into this today, but got a different dependency failing:

# running Build.PL --prefix /nix/store/v9wxq82azcpabd475y8lnw6509hagss9-perl5.30.0-Module-Build-0.4229-x86_64-unknown-linux-musl --installdirs site --config perl=/nix/store/0psxy3wyw9020yr2zmsrlrijrs7sgb7l-perl-5.30.0-x86_64-unknown-linux-musl-dev/bin/perl --fullperl "/nix/store/939gkqx4mbpf5df0s4vx57ii3n79ppm9-perl-5.30.0/bin/perl"
Checking prerequisites...
  test_requires:
    !  Couldn't find a $VERSION in prerequisite File::Temp

Checking optional features...
inc_bundling_support....disabled
  requires:
    ! inc::latest is not installed

ERRORS/WARNINGS FOUND IN PREREQUISITES.  You may wish to install the versions
of the modules indicated above before proceeding with this installation

Attempt to reload Scalar/Util.pm aborted.
Compilation failed in require at /nix/store/aj4caa48nx3ql3mp710sbnm726n4yymr-perl-5.30.0-x86_64-unknown-linux-musl/lib/perl5/5.30.0/x86_64-linux/Data/Dumper.pm line 304.
Couldn't run Build.PL: No such file or directory at lib/Module/Build/Compat.pm line 341.
builder for '/nix/store/29s2b0cav595kpag1855v466cd4rgbf1-perl5.30.0-Module-Build-0.4229-x86_64-unknown-linux-musl.drv' failed with exit code 2

Any idea what that is?

@stale
Copy link

stale bot commented Jun 1, 2020

Thank you for your contributions.
This has been automatically marked as stale because it has had no activity for 180 days.
If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.
Here are suggestions that might help resolve this more quickly:

  1. Search for maintainers and people that previously touched the
    related code and @ mention them in a comment.
  2. Ask on the NixOS Discourse. 3. Ask on the #nixos channel on
    irc.freenode.net.

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 1, 2020
@alyssais alyssais added the 2.status: merge conflict This PR has merge conflicts with the target branch label Aug 16, 2020
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Aug 16, 2020
@ryantm ryantm added 2.status: merge conflict This PR has merge conflicts with the target branch and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Oct 3, 2020
@SuperSandro2000
Copy link
Member

@nh2 Can you please resolve the merge conflict?

@nh2 nh2 marked this pull request as draft November 27, 2020 22:21
@nh2
Copy link
Contributor Author

nh2 commented Nov 27, 2020

@nh2 Can you please resolve the merge conflict?

I'll have to re-test this thoroughly then and also look in detail into the feedback given above; converting back to draft PR for now.

@stale
Copy link

stale bot commented Jun 3, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 3, 2021
@Artturin Artturin closed this Feb 3, 2023
@Artturin
Copy link
Member

Artturin commented Feb 3, 2023

pkgsStatic.git almost builds but fails at t2082-parallel-checkout-attributes.sh so closing this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: merge conflict This PR has merge conflicts with the target branch 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: cross-compilation Building packages on a different platform than they will be used on 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild 10.rebuild-linux: 0 This PR does not cause any packages to rebuild
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants