Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_macos on Github Actions is broken #10110

Closed
straight-shoota opened this issue Dec 20, 2020 · 3 comments
Closed

test_macos on Github Actions is broken #10110

straight-shoota opened this issue Dec 20, 2020 · 3 comments
Labels
kind:bug A bug in the code. Does not apply to documentation, specs, etc. topic:infrastructure

Comments

@straight-shoota
Copy link
Member

straight-shoota commented Dec 20, 2020

test_macos job clearly doesn't work as it is supposed to. It completes successfully even on a broken commit where every other job fails: https://github.com/straight-shoota/crystal/runs/1583371708

I'm not sure what exactly is happening with nix, but it surely doesn't report failure to the job runner.

@straight-shoota straight-shoota added kind:bug A bug in the code. Does not apply to documentation, specs, etc. topic:infrastructure labels Dec 20, 2020
@straight-shoota
Copy link
Member Author

straight-shoota commented Mar 6, 2021

So I looked a bit into the setup of this job, and there seem to be a couple of problems:

  1. In bin/ci build the command nix-shell --pure --run 'TZ=America/New_York make std_spec clean threads=1 junit_output=.junit/std_spec.xml' fails. But the step is not reported as a failure. I don't know why. Does nix-shell silently swallow the error? The CI script certainly errors for on_nix_shell false (Detect failures in Nix build environment #9776 fixed this).
  2. The actual error is /nix/store/r0n4qzr800498l23cq80mg0k03j3aqms-cctools-binutils-darwin-wrapper-927.0.2/nix-support/setup-hook: line 138: ${role_pre}${cmd^^}=${cmd}: bad substitution. Looks like macos ships a really old bash version (3.2) and the substitution requires at least 4.0. So that should be easy to fix, just install a recent bash.
  3. A minor issue: (a) The entire nix environment is only set up in the build stage. So a lot of dependencies are just installed there. I think this should already happen in prepare_build. (b) And ideally it would also be cached. But may that's already happening (there is some cache configuration). I'm not sure what's going on. It feels slow for being cached.

@straight-shoota
Copy link
Member Author

I couldn't actually reproduce the original error on circle-ci's mac runner:

static:project distiller$ bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin18)
Copyright (C) 2007 Free Software Foundation, Inc.
static:project distiller$ CI_NIX_SHELL=true bin/ci build
nix-shell --pure --run 'TZ=America/New_York make std_spec clean threads=1 junit_output=.junit/std_spec.xml'
Using /nix/store/n55dgnwhnc8a6m2q9qhcymvyc0lx3wkg-llvm-10.0.0/bin/llvm-config [version=10.0.0]
...

But I suppose that doesn't really help anything when it still fails on GHA.

@straight-shoota
Copy link
Member Author

#10480 fixes the original error (2) and moves nix initialization to prepare_build stage (3a).

Still need to figure out why the nix error didn't fail the job (1) and clarify if caching works correctly (3b).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:bug A bug in the code. Does not apply to documentation, specs, etc. topic:infrastructure
Projects
None yet
Development

No branches or pull requests

1 participant