From 608df83a50e549e4647f87f6e233768b7b03a4f2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Domen=20Ko=C5=BEar?= Date: Wed, 16 Jun 2021 18:06:05 +0200 Subject: [PATCH] feedback --- source/index.rst | 2 +- source/tutorials/cross-compilation.rst | 119 +++++++++++++++---------- 2 files changed, 74 insertions(+), 47 deletions(-) diff --git a/source/index.rst b/source/index.rst index b2e832dec..083923c0f 100644 --- a/source/index.rst +++ b/source/index.rst @@ -14,7 +14,7 @@ Welcome to nix.dev - Installing software from source code. - Transparent build caching via binary caches. - Strong support for software auditability. - - First-class cross-compilation support. + - First-class cross compilation support. - Remote builds. - Remote deployments. - Atomic upgrades and rollbacks. diff --git a/source/tutorials/cross-compilation.rst b/source/tutorials/cross-compilation.rst index e57ac5e4c..0aa880d5c 100644 --- a/source/tutorials/cross-compilation.rst +++ b/source/tutorials/cross-compilation.rst @@ -1,26 +1,24 @@ .. _ref-cross-compilation: -Cross-compilation +Cross compilation ================= When compiling code, we can distinguish between the **build platform**, where the executable is *built*, and the **host platform**, where the compiled executable *runs*. [#]_ + **Native compilation** is the special case where those two platforms are the same. **Cross compilation** is the general case where those two platforms are not. -It's needed when the host platform has limited resources (such as CPU) +Cross compilation needed when the host platform has limited resources (such as CPU) or when it's not easily accessible for development. -The Nix community has world-class support for cross-compilation, -after years of hard work from our community. +The Nix community has world-class support for cross compilation, +after many years of hard work. -.. [#] Terminology for cross-compilation platforms differs between build systems. +.. [#] Terminology for cross compilation platforms differs between build systems. We have chosen to follow `autoconf terminology `_. -.. note:: macOS/Darwin is a special case, as not the whole OS is open-source. - It's only possible to cross-compile between ``aarch64-darwin`` and ``x86_64-darwin``. - What's a target platform? ------------------------- @@ -31,10 +29,19 @@ It matters in cases where you'd like to distribute a compiler binary, as you'd then like to build a compiler on the build platform, compile code on the host plaform and run the final executable on the target platform. - Since that's rarely needed, we'll treat the target platform the same as the host. +Pinning nixpkgs +--------------- + +To ensure reproducability of this tutorial as explained in :ref:`the pinning tutorial `: + +.. code:: shell-session + + $ NIX_PATH=https://github.com/NixOS/nixpkgs/archive/9420363b95521e65a76eb5153de1eaee4a2e41c6.tar.gz + + Determining the host platform config ------------------------------------ @@ -69,27 +76,37 @@ Some other common examples of platform configs: - x86_64-w64-mingw32 - aarch64-apple-ios +.. note:: macOS/Darwin is a special case, as not the whole OS is open-source. + It's only possible to cross compile between ``aarch64-darwin`` and ``x86_64-darwin``. + ``aarch64-darwin`` support was recently added, so cross compilation is barely tested. + Choosing the host platform with Nix ----------------------------------- Nixpkgs comes with a set of predefined host platforms applied to all packages. -It's possible to list predefined attribute sets via shell completion: +It's possible to explore predefined attribute sets via ``nix repl````: .. code:: shell-session - $ nix-build '' -A pkgsCross. - pkgsCross.aarch64-android pkgsCross.musl32 - pkgsCross.aarch64-android-prebuilt pkgsCross.musl64 - pkgsCross.aarch64be-embedded pkgsCross.muslpi - pkgsCross.aarch64-darwin pkgsCross.musl-power - pkgsCross.aarch64-embedded pkgsCross.or1k - pkgsCross.aarch64-multiplatform pkgsCross.pogoplug4 - pkgsCross.aarch64-multiplatform-musl pkgsCross.powernv - pkgsCross.amd64-netbsd pkgsCross.ppc64 - pkgsCross.arm-embedded pkgsCross.ppc64-musl - pkgsCross.armhf-embedded pkgsCross.ppc-embedded + $ nix repl '' + Welcome to Nix version 2.3.12. Type :? for help. + + Loading ''... + Added 14200 variables. + + nix-repl> pkgsCross. + pkgsCross.aarch64-android pkgsCross.musl-power + pkgsCross.aarch64-android-prebuilt pkgsCross.musl32 + pkgsCross.aarch64-darwin pkgsCross.musl64 + pkgsCross.aarch64-embedded pkgsCross.muslpi + pkgsCross.aarch64-multiplatform pkgsCross.or1k + pkgsCross.aarch64-multiplatform-musl pkgsCross.pogoplug4 + pkgsCross.aarch64be-embedded pkgsCross.powernv + pkgsCross.amd64-netbsd pkgsCross.ppc-embedded + pkgsCross.arm-embedded pkgsCross.ppc64 + pkgsCross.armhf-embedded pkgsCross.ppc64-musl pkgsCross.armv7a-android-prebuilt pkgsCross.ppcle-embedded pkgsCross.armv7l-hf-multiplatform pkgsCross.raspberryPi pkgsCross.avr pkgsCross.remarkable1 @@ -106,25 +123,34 @@ It's possible to list predefined attribute sets via shell completion: pkgsCross.mingw32 pkgsCross.x86_64-netbsd pkgsCross.mingwW64 pkgsCross.x86_64-netbsd-llvm pkgsCross.mmix pkgsCross.x86_64-unknown-redox - pkgsCross.msp430 + pkgsCross.msp430 + +Cross compilation package attribute names are made up, so it isn't always clear +what is the corresponding platform config. +It's possible to query the platform config using:: -Cross-compilation package attribute names are made up, so it isn't always clear -what is the corresponding platform config. + nix-repl> pkgsCross.aarch64-multiplatform.stdenv.hostPlatform.config + "aarch64-unknown-linux-gnu" + +In case the host platform you seek hasn't been defined yet: + +a) `Contribute it upstream `_. + +b) Pass the host platforms to ``crossSystem`` when importing ````:: -It's possible to query the platform config using: + nix-repl> (import { crossSystem = { config = "aarch64-unknown-linux-gnu"; }; }).hello + «derivation /nix/store/qjj23s25kg4vjqq19vxs4dg7k7h214ns-hello-aarch64-unknown-linux-gnu-2.10.drv» - $ nix-instantiate '' -A pkgsCross.aarch64-darwin.hostPlatform.config --eval - "aarch64-apple-darwin" + Or using passing it as an argument to ``nix-build``:: -.. note:: In case the plaform you seek hasn't been defined yet, feel free to contribute one - by `adding it upstream `_. + $ nix-build '' -A hello --arg crossSystem '{ config = "aarch64-unknown-linux-gnu"; }' -Cross-compiling for the first time! +Cross compiling for the first time! ----------------------------------- -To cross-compile a package like `hello `_, +To cross compile a package like `hello `_, pick the platform attribute - ``aarch64-multiplatform`` in our case - and run: .. code:: shell-session @@ -137,11 +163,11 @@ pick the platform attribute - ``aarch64-multiplatform`` in our case - and run: one that you're interested in building. -Real-world cross-compiling of a Hello World example +Real-world cross compiling of a Hello World example --------------------------------------------------- -To show off the power of cross-compilation in Nix, let's build our own Hello World program -by cross-compiling it as static executables to ``armv6l-unknown-linux-gnueabihf`` +To show off the power of cross compilation in Nix, let's build our own Hello World program +by cross compiling it as static executables to ``armv6l-unknown-linux-gnueabihf`` and ``x86_64-w64-mingw32`` (Windows) platforms and run the resulting executable with `an emulator `_. @@ -194,13 +220,14 @@ If we build this example and print both resulting derivations, we should see "He Hello, world! -Developer environment with a cross-compiler +Developer environment with a cross compiler ------------------------------------------- In the :ref:`tutorial for declarative reproducible environments `, we looked at how Nix helps us provide tooling and system libraries for our project. -It's also possible to provide an environment with a compiler configured for cross-compilation. +It's also possible to provide an environment with a compiler configured for **cross-compilation +to static binaries using musl**. Given we have a ``shell.nix``: @@ -210,8 +237,8 @@ Given we have a ``shell.nix``: , pkgs ? (import nixpkgs {}).pkgsCross.aarch64-multiplatform }: - # pkgs.callPackage is needed due to https://github.com/NixOS/nixpkgs/pull/126844 - pkgs.callPackage ({ mkShell, zlib, pkg-config, file }: mkShell { + # callPackage is needed due to https://github.com/NixOS/nixpkgs/pull/126844 + pkgs.pkgsStatic.callPackage ({ mkShell, zlib, pkg-config, file }: mkShell { # these tools run on the build platform, but are configured to target the target platform nativeBuildInputs = [ pkg-config file ]; # libraries needed for the target platform @@ -230,7 +257,7 @@ And ``hello.c``: return 0; } -We can cross-compile it: +We can cross compile it: .. code:: shell-session @@ -241,24 +268,24 @@ And confirm it's aarch64: .. code:: shell-session $ nix-shell --run 'file hello' cross-compile-shell.nix - hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/733hzlw1hixdm6dfdsb8dlwa2h8fl5qi-glibc-2.31-74-aarch64-unknown-linux-gnu/lib/ld-linux-aarch64.so.1, for GNU/Linux 2.6.32, with debug_info, not stripped + hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, with debug_info, not stripped Next steps ---------- -- The `official binary cache `_ doesn't come with binaries - for packages that are cross-compiled, so it's important to set up +- The `official binary cache `_ has very limited number of binaries + for packages that are cross compiled, so to save time recompiling, configure :ref:`a binary cache and CI (GitHub Actions and Cachix) `. -- While many compilers in nixpkgs support cross-compilation, +- While many compilers in nixpkgs support cross compilation, not all of them do. - On top of that, supporting cross-compilation is not trivial + On top of that, supporting cross compilation is not trivial work and due to many possible combinations of what would need to be tested, some packages might not build. - `A detailed explanation how of cross-compilation is implemented in Nix `_ can help with fixing those issues. + `A detailed explanation how of cross compilation is implemented in Nix `_ can help with fixing those issues. - The Nix community has a `dedicated Matrix room `_ - for help around cross-compiling. + for help around cross compiling.