bazel test //examples/xplatform/xctest:xctest fails on MacOS

[jiawen]

I am using Bazel on MacOS Catalina with the bazelbuild/rules_swift at HEAD. I'd like to adopt swift_test for my project but the example itself fails:

$ bazel version
Build label: 3.7.2- (@non-git)
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jan 5 11:43:24 2021 (1609847004)
Build timestamp: 1609847004
Build timestamp as int: 1609847004

$ swiftc -version
Apple Swift version 5.3.2 (swiftlang-1200.0.45 clang-1200.0.32.28)
Target: x86_64-apple-darwin19.6.0

$ git clone https://github.com/bazelbuild/rules_swift.git
$ cd rules_swift/
$ bazel test //examples/xplatform/xctest:xctest
Starting local Bazel server and connecting to it...
INFO: Analyzed target //examples/xplatform/xctest:xctest (35 packages loaded, 955 targets configured).
INFO: Found 1 test target...
FAIL: //examples/xplatform/xctest:xctest (see /private/var/tmp/_bazel_jiawen/12ff43d4eb829d23de5045f50c09e743/execroot/build_bazel_rules_swift/bazel-out/darwin-fastbuild/testlogs/examples/xplatform/xctest/xctest/test.log)
Target //examples/xplatform/xctest:xctest up-to-date:
  bazel-bin/examples/xplatform/xctest/xctest.test-runner.sh
  bazel-bin/examples/xplatform/xctest/xctest.xctest
INFO: Elapsed time: 56.949s, Critical Path: 33.21s
INFO: 216 processes: 6 internal, 209 darwin-sandbox, 1 worker.
INFO: Build completed, 1 test FAILED, 216 total actions
//examples/xplatform/xctest:xctest                                       FAILED in 0.5s
  /private/var/tmp/_bazel_jiawen/12ff43d4eb829d23de5045f50c09e743/execroot/build_bazel_rules_swift/bazel-out/darwin-fastbuild/testlogs/examples/xplatform/xctest/xctest/test.log

INFO: Build completed, 1 test FAILED, 216 total actions

The log shows that no tests are run, just a mysterious error message about a small method list not being in immutable memory, that after some Googling, seems to be a harmless runtime warning.

$ cat /private/var/tmp/_bazel_jiawen/12ff43d4eb829d23de5045f50c09e743/execroot/build_bazel_rules_swift/bazel-out/darwin-fastbuild/testlogs/examples/xplatform/xctest/xctest/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //examples/xplatform/xctest:xctest
-----------------------------------------------------------------------------
objc[30638]: CLASS: class 'examples_xplatform_xctest_xctest.SimpleTests' 0x102e571c0 small method list 0x102e52b98 is not in immutable memory

[jiawen]

Bump.

[keith]

I don't see this same failure locally:

% bazel test //examples/xplatform/xctest:xctest --test_output=all
INFO: Analyzed target //examples/xplatform/xctest:xctest (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //examples/xplatform/xctest:xctest up-to-date:
  bazel-bin/examples/xplatform/xctest/xctest.test-runner.sh
  bazel-bin/examples/xplatform/xctest/xctest.xctest
INFO: Elapsed time: 0.238s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
PASSED: //examples/xplatform/xctest:xctest (see /private/var/tmp/_bazel_ksmiley/7a052ab7f33ef5c3e6513e49d585c877/execroot/build_bazel_rules_swift/bazel-out/darwin-fastbuild/testlogs/examples/xplatform/xctest/xctest/test.log)
INFO: From Testing //examples/xplatform/xctest:xctest
==================== Test output for //examples/xplatform/xctest:xctest:
Test Suite 'All tests' started at 2021-02-12 01:18:59.070
Test Suite 'xctest.xctest' started at 2021-02-12 01:18:59.071
Test Suite 'SimpleTests' started at 2021-02-12 01:18:59.071
Test Case '-[examples_xplatform_xctest_xctest.SimpleTests testThatWillFailIfChanged]' started.
Test Case '-[examples_xplatform_xctest_xctest.SimpleTests testThatWillFailIfChanged]' passed (0.042 seconds).
Test Case '-[examples_xplatform_xctest_xctest.SimpleTests testThatWillSucceed]' started.
Test Case '-[examples_xplatform_xctest_xctest.SimpleTests testThatWillSucceed]' passed (0.000 seconds).
Test Suite 'SimpleTests' passed at 2021-02-12 01:18:59.114.
         Executed 2 tests, with 0 failures (0 unexpected) in 0.042 (0.042) seconds
Test Suite 'xctest.xctest' passed at 2021-02-12 01:18:59.114.
         Executed 2 tests, with 0 failures (0 unexpected) in 0.042 (0.043) seconds
Test Suite 'All tests' passed at 2021-02-12 01:18:59.114.
         Executed 2 tests, with 0 failures (0 unexpected) in 0.042 (0.044) seconds
================================================================================
//examples/xplatform/xctest:xctest                              (cached) PASSED in 1.4s

Executed 0 out of 1 test: 1 test passes.
INFO: Build completed successfully, 1 total action

There's no other output from yours?

[thii]

@jiawen Try to run the test with --macos_minimum_os=<your macOS version or lower>. See #566.

[thii]

Related: bazelbuild/bazel#12988

[philwo]

We've seen this happening since upgrading from Xcode 11.7 to 12.4. Maybe an incompatibility?

[keith]

I'm surprised by the lack of other info here, shouldn't we see something about how the binary fails to launch in this case?

[thii]

https://bugs.swift.org/browse/SR-14013?focusedCommentId=60333&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-60333

Unfortunately, the message can't be fixed. It's emitted by the ObjC runtime, which is part of the OS. The check is gone in newer OS versions, but that doesn't help 10.15.7.
We could conceivably have swift test check the deployment target of the test bundle and error more gracefully when it's higher than the OS being used. Not sure if that's worthwhile, but it certainly could be done.

[keith]

ah nice find

[jiawen]

@jiawen Try to run the test with --macos_minimum_os=<your macOS version or lower>. See #566.

That worked, thanks so much!

I'm fine on closing this issue - unless one of the Bazel maintainers wants to document this elsewhere.