Consider including a pthread.h for all targets

[dicej]

As part of the WASIp2 work, I changed the Makefile to produce a separate include directory for each target, only adding a pthread.h to the wasm32-wasi-threads one (and its wasm32-wasip1-threads alias). That was a change from the old behavior, where we had a single include directory for all targets, with the result that they all had pthread.h.

Lately, I've been helping with the WASI port of .NET, where pthread.h is used in the runtime library for all TARGET_UNIX platforms. That worked fine for older versions of wasi-sdk, since pthread.h was (accidentally?) available for the wasm32-wasi target, but doesn't work for wasi-sdk 22.

We can certainly change the #ifdef TARGET_UNIX conditionals that guard use of pthread.h to e.g. #if (defined TARGET_UNIX) && !(defined TARGET_WASI), but I'm wondering if it would be useful to include some form of pthread.h in the include directories for all targets in wasi-libc. This would make it easier for projects like .NET to upgrade the wasi-sdk version they're using, as well as make it easier to port other software that uses pthread.h but doesn't necessarily ever create more than one thread.

Thoughts?

[sunfishcode]

I think it makes sense to include some form of pthread.h in the include directories for all targets. At one time I had the idea to exclude it so that applications wouldn't detect its presence and enable threads if they wouldn't actually be available, but I now think think that's less important than having pthread.h present to make it easier to port applications.

[sbc100]

Would we also then provide stub versions of the pthread APIs that fail at runtime (when not building with pthread support)?

This is what emscripten does BTW: https://github.com/emscripten-core/emscripten/blob/main/system/lib/pthread/library_pthread_stub.c

[dicej]

Would we also then provide stub versions of the pthread APIs that fail at runtime (when not building with pthread support)?

Probably, yes, although it's interesting that .NET has only needed the header file so far -- I guess it's only using the types and macros in that file, and doesn't need any symbols at link time.

[SingleAccretion]

I guess it's only using the types and macros in that file, and doesn't need any symbols at link time.

That's not quite correct. We had to provide the stubs ourselves as well: https://github.com/dotnet/runtimelab/blob/07ff833c2ab3d441e7f77cd195483fea62667039/src/coreclr/nativeaot/Runtime/wasm/PalRedhawkWasm.cpp#L49-L192.

[ArcaneNibble]

I've been playing with whitequark's port of LLVM to WASI which would need not only/specifically pthread.h but instead the C++ runtime library types such as std::mutex.

In general, LLVM is structured to expect types like std::mutex to at least exist even if threading is disabled (LLVM_ENABLE_THREADS=OFF). It won't try to spawn any threads (after some minor patching) if threads are disabled, and a LTO build in fact won't end up importing wasi::thread-spawn. The resulting build still ends up using atomic WASM opcodes though.

It seems like it'd be nice if no-threads targets simply failed to ever create a thread and emulated atomic operations with non-atomic sequences (which should be safe if no secondary threads can ever be created?)

[whitequark]

@ArcaneNibble Have you been able to build libc++ and LLVM with this patch?

[ArcaneNibble]

I was able to manually build libc++ with threading enabled, shuffle the resulting files around to make a sysroot, and then build your llvm/llvm-project#92677 tree with the result. This was done approximately according to information in https://github.com/YoWASP/clang/blob/main/build.sh but with ad-hoc changes (using the steps for building libc++ first (using a preexisting install of Homebrew clang), cobbling together a WASI sysroot, then using the LLVM build step, and then reusing that sysroot with the resulting WASM build)

I haven't gotten around to figuring out how to properly ship a threading-enabled libc++ as part of the SDK: WebAssembly/wasi-sdk#498