Threaded build broken on Darwin 15.4

[eschnett]

For the past week or so I have been unable to build Julia with threading enabled (JULIA_THREADS=1). Building with threading disabled works fine. This is the last output and error I see:

 clang++ -stdlib=libc++ -mmacosx-version-min=10.7 -m64 -shared -pipe -fPIC -fno-rtti -pedantic  -O3 -g -D_GNU_SOURCE -I. -I/Users/eschnett/src/julia05/src -I/Users/eschnett/src/julia05/src/flisp -I/Users/eschnett/src/julia05/src/support -I/Users/eschnett/src/julia05/usr/include -I/Users/eschnett/src/julia05/usr/include -DLIBRARY_EXPORTS -I/Users/eschnett/src/julia05/deps/valgrind -Wall -Wno-strict-aliasing -fno-omit-frame-pointer -fvisibility=hidden -fno-common -Wpointer-arith -Wundef -I/Users/eschnett/src/julia05/usr/include "-DJL_SYSTEM_IMAGE_PATH=\"../lib/julia/sys.dylib\"" ./jltypes.o ./gf.o ./ast.o ./builtins.o ./module.o ./interpreter.o ./alloc.o ./dlload.o ./sys.o ./init.o ./task.o ./array.o ./dump.o ./toplevel.o ./jl_uv.o ./jlapi.o ./signal-handling.o ./simplevector.o ./APInt-C.o ./runtime_intrinsics.o ./runtime_ccall.o ./threadgroup.o ./threading.o ./stackwalk.o ./gc.o ./codegen.o ./disasm.o ./debuginfo.o ./llvm-simdloop.o ./llvm-gcroot.o -Wl,-rpath,'@loader_path/' -o /Users/eschnett/src/julia05/usr/lib/libjulia.dylib   -Xlinker -all_load ./flisp/libflisp.a -Xlinker -all_load ./support/libsupport.a -L/Users/eschnett/src/julia05/usr/lib -L/Users/eschnett/src/julia05/usr/lib /Users/eschnett/src/julia05/usr/lib/libuv.a /Users/eschnett/src/julia05/usr/lib/libutf8proc.a  -L/Users/eschnett/src/julia05/usr/lib  -lLLVMLTO -lLLVMObjCARCOpts -lLLVMLinker -lLLVMBitWriter -lLLVMIRReader -lLLVMMIRParser -lLLVMAsmParser -lLLVMLibDriver -lLLVMOption -lLLVMDebugInfoPDB -lLLVMTableGen -lLLVMOrcJIT -lLLVMPasses -lLLVMipo -lLLVMVectorize -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMSelectionDAG -lLLVMAsmPrinter -lLLVMX86Desc -lLLVMMCDisassembler -lLLVMX86Info -lLLVMX86AsmPrinter -lLLVMX86Utils -lLLVMMCJIT -lLLVMDebugInfoDWARF -lLLVMLineEditor -lLLVMInterpreter -lLLVMExecutionEngine -lLLVMRuntimeDyld -lLLVMCodeGen -lLLVMTarget -lLLVMScalarOpts -lLLVMProfileData -lLLVMObject -lLLVMMCParser -lLLVMBitReader -lLLVMInstCombine -lLLVMInstrumentation -lLLVMTransformUtils -lLLVMipa -lLLVMMC -lLLVMAnalysis -lLLVMCore -lLLVMSupport -L/Users/eschnett/src/julia05/usr/lib  -lz -lm  -ldl -Wl,-w -framework CoreFoundation -framework CoreServices /Users/eschnett/src/julia05/usr/lib/libosxunwind.a 
ld: section __DATA/__thread_bss extends beyond end of file, file './threading.o' for architecture x86_64

This is on Darwin 15.4.0 (OS X "El Capitan" 10.11.4). I'm building with gcc 5.3 and clang 3.7.1 that were installed via MacPorts (Julia seems to choose clang).

[tkelman]

I think you should be able to force using gcc by setting USEGCC = 1

[eschnett]

@tkelman Is GCC preferred on Darwin?

[tkelman]

No, usually clang is, but it's worth figuring out if this is a clang-specific problem.

[eschnett]

A build with USEGCC=1 fails with the error message

ld: library not found for -lssp

This library is provided by MacPorts GCC.

It seems that Julia overwrites my USEGCC=1 choice and uses clang anyway; at least flisp is still built with clang. The fact that it tries to use ssp means that some other library (libunwind?) probably used gcc.

I tried modifying Make.inc to make Julia use gcc, but that led to errors elsewhere

/Users/eschnett/src/julia05/Make.inc:333: *** USE_LIBCPP only supported with clang. Try setting USE_LIBCPP=0.  Stop.

I think building Julia with gcc on Darwin is not supported.

[tkelman]

"Currently broken" isn't exactly the same thing as "not supported" - I don't see why this couldn't be made to work, and I feel like it used to work at some point not that long ago. It depends exactly how and where you set USEGCC, gmake might be picky about whether or not you used override or whether you used = vs :=.

[yuyichao]

Mac CI build with threading enabled works.

[eschnett]

@yuyichao Do you have a pointer to Travis? I'd like to look at the configurations files, log files, compiler versions etc.

[yuyichao]

https://travis-ci.org/JuliaLang/julia/builds/119249078 (or the tests for the yyc/test/mt branch in general both appveyor and travis) The last test fails because of #15672 but the build of the c files never have any problems.

[staticfloat]

I believe the section __DATA/__thread_bss extends beyond end of file error is a clang-specific error. Specifically, it's a clang 7.3 specific error. See this bug report. I came across it while trying to compile SuiteSparse 4.5.1 on OSX.

[eschnett]

My make output indicates that it's using clang 3.7.1. I also tried with clang 3.8.0 -- same effect. Both were installed via MacPorts.

Clang 7.3 is Apple's clang, isn't it? I didn't use it.

[staticfloat]

If you run clang --version, it might give a version number that is similar to what an Apple clang would give, for example:

$ clang --version
Apple LLVM version 7.3.0 (clang-703.0.29)
Target: x86_64-apple-darwin15.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

I've found that the best way to convince julia to build with GCC instead of clang on OSX is to pass it USECLANG=0 USEGCC=1. You can't give it just one of those overrides, you need both. Also, if you've installed gcc through something like Homebrew or some other package system that doesn't override the builtin gcc name (Which is just a frontend to clang) or is available only via some versioned name such as gcc-5, you need to pass in explicit overrides for not only CC but CXX as well. As such, my full command line to build with gcc 5 from Homebrew instead of clang looks like this:

$ make VERBOSE=1 CC=gcc-5 CXX=g++-5 USECLANG=0 USEGCC=1 USE_LIBCPP=0

[eschnett]

$ clang --version
clang version 3.8.0 (branches/release_38 262722)
Target: x86_64-apple-darwin15.4.0
Thread model: posix
InstalledDir: /opt/local/libexec/llvm-3.8/bin

[eschnett]

A build with gcc with the settings you describe above is accepted by Julia's make system. However, it soon fails with:

libtool: compile:  gcc -m64 -DPACKAGE_NAME=\"libuv\" -DPACKAGE_TARNAME=\"libuv\" -DPACKAGE_VERSION=\"0.11.26\" "-DPACKAGE_STRING=\"libuv 0.11.26\"" -DPACKAGE_BUGREPORT=\"https://github.com/joyent/libuv/issues\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libuv\" -DVERSION=\"0.11.26\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_LIBDL=1 -DHAVE_LIBPTHREAD=1 -DHAVE_DTRACE=1 -I. -I/Users/eschnett/src/julia05/deps/srccache/libuv-07730c4bd595b4d45a498a8ee0bcd53878ff7c10 -I/Users/eschnett/src/julia05/deps/srccache/libuv-07730c4bd595b4d45a498a8ee0bcd53878ff7c10/include -I/Users/eschnett/src/julia05/deps/srccache/libuv-07730c4bd595b4d45a498a8ee0bcd53878ff7c10/src -I/Users/eschnett/src/julia05/deps/srccache/libuv-07730c4bd595b4d45a498a8ee0bcd53878ff7c10/src/unix -g -O2 -D_DARWIN_USE_64_BIT_INODE=1 -g -O2 -MT src/unix/libuv_la-async.lo -MD -MP -MF src/unix/.deps/libuv_la-async.Tpo -c /Users/eschnett/src/julia05/deps/srccache/libuv-07730c4bd595b4d45a498a8ee0bcd53878ff7c10/src/unix/async.c  -fno-common -DPIC -o src/unix/.libs/libuv_la-async.o
In file included from /usr/include/Availability.h:172:0,
                 from /usr/include/sys/stat.h:75,
                 from /Users/eschnett/src/julia05/deps/srccache/libuv-07730c4bd595b4d45a498a8ee0bcd53878ff7c10/include/uv-unix.h:26,
                 from /Users/eschnett/src/julia05/deps/srccache/libuv-07730c4bd595b4d45a498a8ee0bcd53878ff7c10/include/uv.h:64,
                 from /Users/eschnett/src/julia05/deps/srccache/libuv-07730c4bd595b4d45a498a8ee0bcd53878ff7c10/src/unix/async.c:25:
/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Headers/LSInfoDeprecated.h:1242:34: error: expected ',' or '}' before '__attribute__'
   kLSHandlerOptionsDefault       __OSX_AVAILABLE_BUT_DEPRECATED_MSG(__MAC_10_4, __MAC_10_11, __IPHONE_4_0, __IPHONE_9_0, "Creator codes are deprecated on OS X.") = 0,
                                  ^
/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Headers/LSOpenDeprecated.h:27:33: error: expected ',' or '}' before '__attribute__'
   kLSLaunchStartClassic         __OSX_AVAILABLE_BUT_DEPRECATED_MSG(__MAC_10_0, __MAC_10_11, __IPHONE_NA, __IPHONE_NA, "The Classic environment is no longer supported.") = 0x00020000, /* Does nothing.*/
                                 ^
make[3]: *** [src/unix/libuv_la-async.lo] Error 1
make[2]: *** [all] Error 2
make[1]: *** [build/libuv-07730c4bd595b4d45a498a8ee0bcd53878ff7c10/.libs/libuv.la] Error 2
make: *** [julia-deps] Error 2

That's a known incompatibility between Darwin's system header files and newer versions of gcc. The system header files use a non-standard extension of the C standard (they use __attribute__ for enums).

[eschnett]

To work around this issue, I downgraded Xcode from 7.3 to 7.2. Everything is working fine again.

The 7.2 download is available from Apple's developer web site. The downgrade itself is straightforward.

[kpamnany]

I've just run into this also, but I don't want to downgrade or muck around with gcc. Hoping for a fix. :)

[andreasnoack]

@kpamnany See #15767.