Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added environment variables to control the UID and GID in the container #543

Merged
merged 1 commit into from
Mar 29, 2021

Conversation

UebelAndre
Copy link
Contributor

CROSS_CONTAINER_UID and CROSS_CONTAINER_GID can now be used to control the user and group ID's passed to Docker.

I ran into this when trying to setup a Github-Action that runs on MacOS and uses docker-machine which wasn't working. I then found #389 which described the solution of updating the UID and GID which I've confirmed works. This change should make it easier for users to solve for the same issue without making a fork.

@UebelAndre UebelAndre requested review from Dylan-DPC-zz and a team as code owners March 29, 2021 01:48
@UebelAndre
Copy link
Contributor Author

UebelAndre commented Mar 29, 2021

I'm not sure of others would find this useful, but I ran into this when trying to setup a github-action that would cross-compile binaries for me and commit them back to the repo. The trickiest part was probably getting docker to work but is then what lead me to this PR since I had to use docker-machine.

---
name: CI+CD
on:
  push:
    branches:
      - main

jobs:
  crate_universe:
    runs-on: macos-10.15
    steps:
      - uses: actions/checkout@v2
        with:
          # Save the git credentials so we can commit back to the repo
          persist-credentials: true
      - run: |
          # Install docker for use with cross
          # https://medium.com/crowdbotics/a-complete-one-by-one-guide-to-install-docker-on-your-mac-os-using-homebrew-e818eb4cfc3
          brew install docker docker-machine boot2docker boot2docker-cli
      - run: |
          # Configure docker-machine
          docker-machine create --driver virtualbox --virtualbox-boot2docker-url https://github.com/boot2docker/boot2docker/releases/download/v19.03.12/boot2docker.iso default
          # For some reason, `sed -r 's/export (.*)="(.*)"/\1=\2/'` does not work because `-r` is not a valid option for sed
          docker-machine env default | grep "export" | sed 's/export //' | sed 's/"//' | sed 's/"//' >> $GITHUB_ENV
          echo "COMPOSE_INTERACTIVE_NO_CLI=0" >> $GITHUB_ENV
      - run: |
          # Confirm docker is runnable
          docker --version
          # Ensure docker works and volumes are able to be used to write back to the host
          docker run --rm -v "${HOME}:/root" alpine touch /root/hello-from-docker
          rm ~/hello-from-docker
      - run: |
          # Install cross
          #curl -Lo ~/cross.tar.gz https://github.com/rust-embedded/cross/releases/download/v0.2.1/cross-v0.2.1-x86_64-apple-darwin.tar.gz
          #tar -xf ~/cross.tar.gz -C /usr/local/bin/
          #chmod +x /usr/local/bin/cross

          # Install a version that solves for https://github.com/rust-embedded/cross/pull/543
          cargo install --git https://github.com/rust-embedded/cross.git --branch master
          echo "CROSS_CONTAINER_UID=1000" > $GITHUB_ENV
          cross --version
      - run: |
          # Install host toolchains
          rustup update stable && rustup default stable
      - run: |
          # Setup macos build tooling
          sudo xcode-select -s /Applications/Xcode_12.4.app/Contents/Developer/

          # Set SDK environment variables
          echo "SDKROOT=$(xcrun -sdk macosx11.1 --show-sdk-path)" >> $GITHUB_ENV
          echo "MACOSX_DEPLOYMENT_TARGET=$(xcrun -sdk macosx11.1 --show-sdk-platform-version)" >> $GITHUB_ENV
      - run: |
          # Build all binaries
          ./bootstrap.sh
      - run: |
          # Stage and display the changes
          git stage -A . && git status
      - run: |
          # Setup git and commit back to the repo
          git config user.email "uebel_andre@uebelandre.com" 
          git config user.name "uebel_andre"
          git commit -m 'github-actions: auto-updated binaries'
          git push origin

For more context on the level of complexity in this action around setting up docker, see actions/runner-images#115

@UebelAndre
Copy link
Contributor Author

@Dylan-DPC Hey, do you have a quick moment to take a look at this? 🙏

@Dylan-DPC-zz
Copy link

bors: r+

@bors
Copy link
Contributor

bors bot commented Mar 29, 2021

Build succeeded:

@bors bors bot merged commit 58bfd65 into cross-rs:master Mar 29, 2021
@UebelAndre UebelAndre deleted the user branch March 29, 2021 18:32
@UebelAndre UebelAndre restored the user branch March 29, 2021 18:32
@UebelAndre
Copy link
Contributor Author

@Dylan-DPC Would it be possible to get a new release with this commit as well?

@Dylan-DPC-zz
Copy link

@UebelAndre unfortunately I don't have permissions for that and someone else would have to do it

@UebelAndre UebelAndre mentioned this pull request Mar 29, 2021
@TarekkMA
Copy link
Contributor

TarekkMA commented Apr 1, 2021

@UebelAndre, I did manage to run docker on GH actions using vagrant. but I get this error.

"/usr/local/bin/docker" "run" "--userns" "host" "-e" "RUST_BACKTRACE" "-e" "RUST_LOG" "-e" "RUST_DEBUG" "-e" "DONT_RUSTFMT" "-v" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rsdroid-testing/assets:/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rsdroid-testing/assets" "-e" "OUT_DIR=/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rsdroid-testing/assets" "-e" "PKG_CONFIG_ALLOW_CROSS=1" "--rm" "--user" "1000:20" "-e" "XARGO_HOME=/xargo" "-e" "CARGO_HOME=/cargo" "-e" "CARGO_TARGET_DIR=/target" "-e" "USER=runner" "-e" "CROSS_RUNNER=" "-v" "/Users/runner/.xargo:/xargo:Z" "-v" "/Users/runner/.cargo:/cargo:Z" "-v" "/cargo/bin" "-v" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge://Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge:Z" "-v" "/Users/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:Z,ro" "-v" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge/target:/target:Z" "-w" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge" "ankidroid/rslib-bridge-linux" "sh" "-c" "PATH=$PATH:/rust/bin cargo build --release --features no-android --verbose --target x86_64-unknown-linux-gnu"
sh: cargo: command not found

It seems like the the /rust/bin is empty

@UebelAndre
Copy link
Contributor Author

@UebelAndre, I did manage to run docker on GH actions using vagrant. but I get this error.

"/usr/local/bin/docker" "run" "--userns" "host" "-e" "RUST_BACKTRACE" "-e" "RUST_LOG" "-e" "RUST_DEBUG" "-e" "DONT_RUSTFMT" "-v" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rsdroid-testing/assets:/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rsdroid-testing/assets" "-e" "OUT_DIR=/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rsdroid-testing/assets" "-e" "PKG_CONFIG_ALLOW_CROSS=1" "--rm" "--user" "1000:20" "-e" "XARGO_HOME=/xargo" "-e" "CARGO_HOME=/cargo" "-e" "CARGO_TARGET_DIR=/target" "-e" "USER=runner" "-e" "CROSS_RUNNER=" "-v" "/Users/runner/.xargo:/xargo:Z" "-v" "/Users/runner/.cargo:/cargo:Z" "-v" "/cargo/bin" "-v" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge://Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge:Z" "-v" "/Users/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:Z,ro" "-v" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge/target:/target:Z" "-w" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge" "ankidroid/rslib-bridge-linux" "sh" "-c" "PATH=$PATH:/rust/bin cargo build --release --features no-android --verbose --target x86_64-unknown-linux-gnu"
sh: cargo: command not found

It seems like the the /rust/bin is empty

I'm not familiar with the error you ran into. I ended up doing exactly what you see in #543 (comment) and successfully built my project using cross. You might need some steps like

- run: |
          # Install host toolchains
          rustup update stable && rustup default stable

But I'm no expert here. My recommendation is to use what I wrote since I was able to confirm it works for simple use cases.

@UebelAndre UebelAndre deleted the user branch April 1, 2021 00:10
@TarekkMA
Copy link
Contributor

TarekkMA commented Apr 1, 2021

Thank you, will try that

@TarekkMA
Copy link
Contributor

TarekkMA commented Apr 1, 2021

@UebelAndre sorry for pinging you again.

Did you encounter any spurious network error or mmap errors?

Full log: https://github.com/david-allison-1/Anki-Android-Backend/runs/2244327761

bash --login -c CC=$ANKIDROID_LINUX_CC DONT_RUSTFMT=1 cross build --release --features no-android --verbose  --target x86_64-unknown-linux-gnu
+ "rustc" "--print" "sysroot"
+ "rustup" "toolchain" "list"
+ "rustup" "toolchain" "add" "stable-x86_64-unknown-linux-gnu"
info: installing component 'cargo'
info: using up to 500.0 MiB of RAM to unpack components
info: installing component 'rust-std'
info: installing component 'rustc'

  stable-x86_64-unknown-linux-gnu installed - (error reading rustc version)

+ "rustup" "target" "list" "--toolchain" "stable-x86_64-unknown-linux-gnu"
+ "rustup" "component" "list" "--toolchain" "stable-x86_64-unknown-linux-gnu"
info: downloading component 'rust-src'
+ "rustup" "component" "add" "rust-src" "--toolchain" "stable-x86_64-unknown-linux-gnu"
info: installing component 'rust-src'
info: using up to 500.0 MiB of RAM to unpack components
+ "/usr/local/bin/docker" "run" "--userns" "host" "-e" "RUST_BACKTRACE" "-e" "RUST_LOG" "-e" "RUST_DEBUG" "-e" "DONT_RUSTFMT" "-e" "PKG_CONFIG_ALLOW_CROSS=1" "--rm" "--user" "1000:20" "-e" "XARGO_HOME=/xargo" "-e" "CARGO_HOME=/cargo" "-e" "CARGO_TARGET_DIR=/target" "-e" "USER=runner" "-e" "CROSS_RUNNER=" "-v" "/Users/runner/.xargo:/xargo:Z" "-v" "/Users/runner/.cargo:/cargo:Z" "-v" "/cargo/bin" "-v" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge://Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge:Z" "-v" "/Users/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:Z,ro" "-v" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge/target:/target:Z" "-w" "/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge" "ankidroid/rslib-bridge-linux" "sh" "-c" "PATH=$PATH:/rust/bin cargo build --release --features no-android --verbose --target x86_64-unknown-linux-gnu"
    Updating git repository `https://github.com/ankitects/rust-coarsetime.git`
warning: spurious network error (2 tries remaining): failed to mmap. Could not write data: Invalid argument; class=Os (2)
warning: spurious network error (1 tries remaining): failed to mmap. Could not write data: Invalid argument; class=Os (2)
error: failed to get `coarsetime` as a dependency of package `anki v2.1.34 (/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge/anki/rslib)`

    ... which is depended on by `rsdroid v0.1.0 (/Users/runner/work/Anki-Android-Backend/Anki-Android-Backend/rslib-bridge)`
> Task :rsdroid-testing:preBuildLinux FAILED

Caused by:
  failed to load source for dependency `coarsetime`

Caused by:
  Unable to update https://github.com/ankitects/rust-coarsetime.git?branch=old-mac-compat#f9e2c862

Caused by:
  failed to clone into: /cargo/git/db/rust-coarsetime-ca006422e88c3421

Caused by:
  failed to mmap. Could not write data: Invalid argument; class=Os (2)

@UebelAndre
Copy link
Contributor Author

@UebelAndre sorry for pinging you again.

Did you encounter any spurious network error or mmap errors?

Full log: https://github.com/david-allison-1/Anki-Android-Backend/runs/2244327761

I did not, everything ran quite smoothly for me. Though, I think related to another issue, I had to cargo clean before each target was built (which I built in a loop).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants