Wrong architecture for binaries downloaded on ARM Macs

[deviantintegral]

The problem

On ARM Macs, running vendor/bin/task test:static gives the following error under Docker Desktop:

$ vendor/bin/task test:static
task: [test:phpunit:static] CONFIG="vendor/lullabot/drainpipe-dev/scaffold/phpunit.xml"
if [ "" == "true" ]; then
  CONFIG="vendor/lullabot/drainpipe-dev/scaffold/phpunit-testtraits.xml"
fi
if [ "" == "junit" ]; then
  mkdir -p test_result
  ./vendor/bin/phpunit --testsuite=unit -c $CONFIG --log-junit test_result/phpunit.xml
else
  ./vendor/bin/phpunit -c $CONFIG --testsuite=unit
fi

task: [test:phpstan] DIRS_ARR=(web/modules/custom web/themes/custom web/sites)
for DIR in "${DIRS_ARR[@]}"; do
  mkdir -p $DIR
done


task: [test:security] if [ "" == "junit" ]; then
  mkdir -p test_result
  ./vendor/bin/local-php-security-checker --format=json > test_result/local-php-security-checker.json
fi

task: [test:lint] composer validate
task: [test:phpcs] DIRS_ARR=(web/modules/custom web/themes/custom web/sites)
for DIR in "${DIRS_ARR[@]}"; do
  mkdir -p $DIR
done

if [ "" == "junit" ]; then
  mkdir -p test_result
  ./vendor/bin/phpcs --report=junit -q > test_result/phpcs.xml
else
  ./vendor/bin/phpcs
fi

task: [test:security] if [ "" == "junit" ]; then
  RESULT=$(cat test_result/local-php-security-checker.json)
  ./vendor/bin/drainpipe-convert-to-junit-xml convert:sensiolabs-security-check "$RESULT" > test_result/local-php-security-checker.xml
  if [ "$RESULT" != "{}" ]; then
    exit 1
  fi
else
  ./vendor/bin/local-php-security-checker
fi

task: [test:phpstan] if [ "" == "junit" ]; then
  mkdir -p test_result
  ./vendor/bin/phpstan analyse --error-format=junit web/modules/custom web/themes/custom web/sites > test_result/phpstan.xml
else
  ./vendor/bin/phpstan analyse web/modules/custom web/themes/custom web/sites
fi

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x4f0 pc=0x808a22f]

runtime stack:
runtime.throw({0x834e7c2, 0x2a})
	/opt/hostedtoolcache/go/1.18.6/x64/src/runtime/panic.go:992 +0x6a
runtime.sigpanic()
	/opt/hostedtoolcache/go/1.18.6/x64/src/runtime/signal_unix.go:802 +0x233
runtime.runqput(0x0, 0x88004b0, 0x1)
	/opt/hostedtoolcache/go/1.18.6/x64/src/runtime/proc.go:5686 +0xaf
runtime.newproc.func1()
	/opt/hostedtoolcache/go/1.18.6/x64/src/runtime/proc.go:4059 +0x56
runtime.systemstack()
	/opt/hostedtoolcache/go/1.18.6/x64/src/runtime/asm_386.s:370 +0x41

goroutine 1 [running, locked to thread]:
	goroutine running on other thread; stack unavailable
task: Failed to run task "test:security": exit status 2

Under colima we get a different error:

$ vendor/bin/task test:static
task: [test:phpcs] DIRS_ARR=(web/modules/custom web/themes/custom web/sites)
for DIR in "${DIRS_ARR[@]}"; do
  mkdir -p $DIR
done

if [ "" == "junit" ]; then
  mkdir -p test_result
  ./vendor/bin/phpcs --report=junit -q > test_result/phpcs.xml
else
  ./vendor/bin/phpcs
fi

task: [test:phpstan] DIRS_ARR=(web/modules/custom web/themes/custom web/sites)
for DIR in "${DIRS_ARR[@]}"; do
  mkdir -p $DIR
done


task: [test:lint] composer validate
task: [test:phpunit:static] CONFIG="vendor/lullabot/drainpipe-dev/scaffold/phpunit.xml"
if [ "" == "true" ]; then
  CONFIG="vendor/lullabot/drainpipe-dev/scaffold/phpunit-testtraits.xml"
fi
if [ "" == "junit" ]; then
  mkdir -p test_result
  ./vendor/bin/phpunit --testsuite=unit -c $CONFIG --log-junit test_result/phpunit.xml
else
  ./vendor/bin/phpunit -c $CONFIG --testsuite=unit
fi

task: [test:security] if [ "" == "junit" ]; then
  mkdir -p test_result
  ./vendor/bin/local-php-security-checker --format=json > test_result/local-php-security-checker.json
fi

task: [test:security] if [ "" == "junit" ]; then
  RESULT=$(cat test_result/local-php-security-checker.json)
  ./vendor/bin/drainpipe-convert-to-junit-xml convert:sensiolabs-security-check "$RESULT" > test_result/local-php-security-checker.xml
  if [ "$RESULT" != "{}" ]; then
    exit 1
  fi
else
  ./vendor/bin/local-php-security-checker
fi

task: Failed to run task "test:security": fork/exec /var/www/html/vendor/bin/local-php-security-checker: exec format error

We can see from the first message that the x64 version of task is being used, but replacing task didn't fix it. But the colima clarifies something - there's another binary that's the wrong format. Let's identify them:

$ file vendor/bin/* | grep -v 'ASCII text'
vendor/bin/local-php-security-checker:                 symbolic link to local-php-security-checker_2.0.5_linux_386
vendor/bin/local-php-security-checker_2.0.5_linux_386: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, Go BuildID=qJeMo4nq4Cuq5VQ3qBb-/cPB1TOoEhzdpN5NPiwMQ/j_-1BDU5e9eW6BzoxzuO/6el-aBLzwC5LPVio2ez8, stripped
vendor/bin/task:                                       ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=gV9Qh1fAmFziNalbQ5qJ/pSzrlUDBsL6hU61eGqGE/13aKCApAYH2nqf6uH5jj/D_aJUZ-LOt4SYnEjTks-, stripped

Woah - local-php-security-checker is coming in as a 32-bit executable. This makes a lot more sense. 64-bit x86 is mostly an extension of the 32-bit instruction set, so it makes some sense that one of the environments is able to execute it somewhat. I'm guessing the kernel shipped with Docker Desktop has some level of 32-bit support, while colima has none.

Architecture is determined at https://github.com/Lullabot/drainpipe/blob/main/src/BinaryInstaller.php#L65

On Docker Desktop, there's no architecture:

$ php -r 'print php_uname("v") . "\n";'
#1 SMP PREEMPT Thu Jun 30 08:18:26 UTC 2022

On colima no architecture is shown at all, causing us to fall to the default case of 386:

$ php -r 'print php_uname("v") . "\n";'
#1-Alpine SMP Fri, 16 Sep 2022 06:29:31 +0000

Proposed Fix

1. Fix the uname call to be php_uname('m');
2. I see aarch64 being emitted by that in both Docker Desktop and Colima. We need to change or add the architecture check from the current arm64.
3. I think we should explicitly elseif the check for 386 instead of making it a default case.
4. I think with using 'm' we can avoid the \PHP_INT_SIZE check.
5. I think a default case should be 64-bit intel, while also emitting a warning on composer install, or perhaps erroring out entirely.
6. Given how painful this was to debug and figure out, I think we should add tests around this with GitHub actions. We can at least add jobs to run on x64 and ARM and verify the correct binaries are downloaded, even if we can't easily test the 386 versions.

[deviantintegral]

Related issues: #113 and #114

[deviantintegral]

We're getting occasional segfaults with task. Reported by colleagues at Webspec:

$ ddev task refresh site=@iowa-ministry-of-magic
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
/mnt/ddev_config/commands/web/task: line 7:  2300 Segmentation fault      ./vendor/bin/task "$@"

[deviantintegral]

@deviantintegral to retest this.

[deviantintegral]

I'm not getting segfaults, but I am getting odd hangs in task. It will just completely halt until I kill it. This totally fits in with "not quite reliable emulation", so while it may not be the root cause getting this known issue fixed will at least rule it out.

[baluv3]

Noting here that we think this issue will be resolved, when #172 is completed.