Skip to content

Latest commit

 

History

History
255 lines (198 loc) · 8.21 KB

build.md

File metadata and controls

255 lines (198 loc) · 8.21 KB
Raw

Building Mayastor

You will not need to always build Mayastor. We will provide official x86_64 & aarch64 binaries and images in future releases.

Mayastor is a multi-component Rust project that makes heavy use use of Nix for our development and build process.

If you're coming from a non-Rust (or non-Nix) background, building Mayastor may be a bit different than you're used to. There is no Makefile, you won't need a build toolchain, you won't need to worry about cross compiler toolchains, and all builds are reproducable.

Table of Contents

Prerequisites

Mayastor only builds on modern Linuxes. We'd adore contributions to add support for Windows, FreeBSD, OpenWRT, or other server platforms.

If you do not have Linux system:

  • Windows: We recommend using WSL2 if you only need to build Mayastor. You'll need a Hyper-V VM if you want to use it.
  • Mac: We recommend you use Docker for Mac and follow the Docker process described. Please let us know if you find a way to run it!
  • FreeBSD: We think this might actually work, SPDK is compatible! But, we haven't tried it yet.
  • Others: This is kind of a "Do-it-yourself" situation. Sorry, we can't be more help!

The only thing your system needs to build Mayastor is Nix.

Usually Nix can be installed via (Do not use sudo!):

curl -L https://nixos.org/nix/install | sh

Can't install Nix?

That's totally fine. You can use docker just fine for one-off or occasional PRs!

This flow will get you a pre-fetched nix store:

docker run --name mayastor-nix-prefetch -it -v $(pwd):/scratch:rw --privileged --workdir /scratch nixos/nix nix-shell --run "exit 0"
docker commit mayastor-nix-prefetch mayastor/dev-env:latest
docker rm mayastor-nix-prefetch
docker run --rm -it -v $(pwd):/scratch:rw --workdir /scratch mayastor/dev-env:latest nix-shell

To re-enter, just run the last command again.

  • Some of our team uses NixOS which has nix baked in, but you don't need to.
  • Some of our team uses `direnv, but you don't need to.

For some tasks, we use features from nixUnstable. You can use nixos-unstable (or nixpkgs-unstable for nix users) by changing your channel.

First, setting the following:

{ pkgs, ... }: {
   nix.extraOptions = ''
      experimental-features = nix-command flakes
   '';
   nix.package = pkgs.nixUnstable;
}

Then, updating the channel:

$ sudo nix-channel --list
nixos https://nixos.org/channels/nixos-20.09
$ sudo nix-channel --remove nixos
$ sudo nix-channel --add https://nixos.org/channels/nixos-unstable nixos
$ sudo nixos-rebuild switch --update

If you don't want, you can drop into a nixUnstable supporting shell with:

nix-shell -I nixpkgs=channel:nixpkgs-unstable -p nixUnstable --command "nix --experimental-features 'nix-command flakes' develop -f . mayastor"

Don't want to use nixUnstable? That's ok! Use nix-shell and nix-build as you normally would.

Want to run or hack on Mayastor? You need more configuration! See running, then testing.

You can use a tool like direnv to automate nix shell entry. If you are unable to use the Nix provided Rust for some reason, there are norust and nospdk arguments to Nix shell. nix-shell --arg norust true

Iterative Builds

Contributors often build Mayastor repeatedly during the development process. Using nix develop to enter a more persistent development shell can help improve iteration time:

nix develop -f . mayastor

Once entered, you can start any tooling (eg code .) to ensure the correct resources are available. The project can then be interacted with like any other Rust project.

Building:

cargo build
cargo build --release

Want to run or hack on Mayastor? You need more configuration! See [running][doc-running], then [testing][doc-testing].

To ensure you are aware of this, we greet you with a nice cow.

Artifacts

There are a few ways to build Mayastor! If you're hacking on Mayastor, it's best to use nix develop (above) then turn to traditional Rust tools. If you're looking for releases, use nix build or nix bundle depending on your needs.

Why is the build process this way?

Mayastor creates reproducable builds, it won't use any of your local system dependencies (other than nix). This is a component of the best practices of the Core Infrastructure Initiative. More on how Nix works can be found in the Nix paper.

Building non-portable Nix derivations

You can build release binaries of Mayastor with nix build:

for PKG in mayastor; do
  echo "Building ${PKG} to artifacts/pkgs/${PKG}"; \
  nix build -f . -o artifacts/pkgs/${PKG} ${PKG};
done

Try them as if they were installed:

nix shell -f . mayastor

Building portable Nix bundles

In order to make an artifact which can be distributed, we use nix bundle.

TODO: We currently don't generate bundles some executables, such as io-engine-client. This is coming.

for BUNDLE in mayastor; do
  echo "Bundling ${BUNDLE} to artifacts/bundle/${BUNDLE}"; \
  nix bundle -f . -o artifacts/bundles/${BUNDLE} ${BUNDLE};
done

Test them:

for FILE in artifacts/bundles/*; do
 echo "Testing bundle ${FILE}..."
 ${FILE} --version
done

Building Docker images

Build the Docker images with nix build:

for IMAGE in \
  io-engine-client mayastor mayastor-csi io-engine-client kiiss-service \
  node-service volume-service pool-service rest-service node-operator; \
do
  echo "Building ${IMAGE} to artifacts/docker/${IMAGE}.tar"; \
  nix build -f . -o artifacts/docker/${IMAGE}.tar images.${IMAGE};
done

Optionally, the generated Docker images will not tag to latest. You may wish to do that if you want to run them locally:

for FILE in artifacts/docker/*.tar; do
 echo "Loading ${FILE}..."
 docker load --quiet --input ${FILE} \
   | awk '{ print $3 }' \
   | ( \
      read IMAGE; \
      LATEST=$(echo ${IMAGE} | awk '{split($0,a,":"); print a[1]}'):latest; \
      echo "Tagging ${IMAGE} to ${LATEST} (from ${FILE})."; \
      docker tag ${IMAGE} ${LATEST}; \
     );
done

Then, to test the images:

for FILE in artifacts/docker/*.tar; do
  echo "Loading ${FILE}..."
  docker load --quiet --input ${FILE} \
    | awk '{ print $3 }' \
    | ( \
        read IMAGE; \
        echo "Testing ${IMAGE} (from ${FILE})."; \
        docker run --rm --interactive ${IMAGE} --version; \
        docker rmi ${IMAGE} > /dev/null
      );
done

Building KVM images

TODO: We're still writing this! Sorry!

Building Artifacts the Hard Way

This isn't really the 'hard way', you'll still use cargo.

TODO: We're still writing this! Sorry! For now, please refer to spdk-rs README on this matter.