Add docker image that can be used to build projects with wasi-sdk sysroot #271

anuraaga · 2022-12-19T06:09:53Z

Fixes #176

Based on the issue it seems there is interest in this with some help, so sent a PR to add building and publishing of a wasi-sdk docker image. emscripten provides an official emsdk docker image to make it easier to build existing projects to webassembly, for example wabt uses it to build their emscripten artifact. A similar docker image for wasi-sdk can allow C++ projects to more easily provide webassembly for use outside the browser - for example, building wabt and binaryen to wasi will allow full webassembly toolchains without the need for installing multiple tools or C toolchains.

The docker image is built from main pushes with main and the github sha as tags, and for tag pushes with the tag name. You can see what it looks like in my fork https://github.com/anuraaga/wasi-sdk/pkgs/container/wasi-sdk

Using ghcr allows publishing docker with no worries of account or secret management. In the future, it could also be published to Docker Hub if desired but I personally don't see much point in OSS projects doing so anymore.

I have confirmed that wabt (cmake project) and re2 (make project) build fine with this image.

This approach does not use the LLVM built with wasi-sdk but instead installs packages and configures environment variables to use the wasi sysroot / target. This is to support multiplatform docker images - with arm64 becoming more popular it is important to provide images that work with it. While it's possible to build arm64 with qemu on GitHub Actions, for a monster like LLVM it's impractical.

autotools, cmake and ninja are also included to represent a common set of build tools used by C++ projects out there - the more included the bigger the image gets, but this feels like good coverage.

Let me know what you think

…root

…asi-sdk-docker

.gitignore

sdk.Dockerfile

sbc100 · 2022-12-19T12:07:17Z

sdk.Dockerfile

+\
+    curl -sS https://apt.llvm.org/llvm-snapshot.gpg.key | gpg --dearmor > /etc/apt/trusted.gpg.d/llvm.gpg && \
+    echo "deb [signed-by=/etc/apt/trusted.gpg.d/llvm.gpg] http://apt.llvm.org/jammy/ llvm-toolchain-jammy-${LLVM_VERSION} main" >> /etc/apt/sources.list.d/llvm.list && \
+    echo "deb-src [signed-by=/etc/apt/trusted.gpg.d/llvm.gpg] http://apt.llvm.org/jammy/ llvm-toolchain-jammy-${LLVM_VERSION} main" >> /etc/apt/sources.list.d/llvm.list && \


Can we maybe just use the llvm that ships with jammy? Is 13.0 recent enough?

It would worry me to use a version of LLVM older than what's published with wasi-sdk for potential incompatibilities, currently it looks like wasi-sdk is on 15.0.6

sdk.docker.cmake

sbc100 · 2022-12-19T12:13:41Z

sdk.docker.cmake

+set(CMAKE_C_COMPILER /usr/bin/clang-15)
+set(CMAKE_CXX_COMPILER /usr/bin/clang++-15)
+set(CMAKE_AR /usr/bin/llvm-ar-15)
+set(CMAKE_RANLIB /usr/bin/llvm-ranlib-15)


Given that you set CC/CXX etc in the environment maybe this is redundant here?

With cmake, CC/CXX do get sourced from env variables, but AR RANLIB still don't unfortunately. It makes the state clearest I think to have it all in one place here rather than relying on environment for some but not all

anuraaga

Thanks @sbc100 - applied some cleanups and added some responses

sdk.Dockerfile

anuraaga · 2022-12-20T01:43:12Z

sdk.Dockerfile

+\
+    curl -sS https://apt.llvm.org/llvm-snapshot.gpg.key | gpg --dearmor > /etc/apt/trusted.gpg.d/llvm.gpg && \
+    echo "deb [signed-by=/etc/apt/trusted.gpg.d/llvm.gpg] http://apt.llvm.org/jammy/ llvm-toolchain-jammy-${LLVM_VERSION} main" >> /etc/apt/sources.list.d/llvm.list && \
+    echo "deb-src [signed-by=/etc/apt/trusted.gpg.d/llvm.gpg] http://apt.llvm.org/jammy/ llvm-toolchain-jammy-${LLVM_VERSION} main" >> /etc/apt/sources.list.d/llvm.list && \


It would worry me to use a version of LLVM older than what's published with wasi-sdk for potential incompatibilities, currently it looks like wasi-sdk is on 15.0.6

anuraaga · 2022-12-20T01:46:50Z

sdk.docker.cmake

+set(CMAKE_C_COMPILER /usr/bin/clang-15)
+set(CMAKE_CXX_COMPILER /usr/bin/clang++-15)
+set(CMAKE_AR /usr/bin/llvm-ar-15)
+set(CMAKE_RANLIB /usr/bin/llvm-ranlib-15)


With cmake, CC/CXX do get sourced from env variables, but AR RANLIB still don't unfortunately. It makes the state clearest I think to have it all in one place here rather than relying on environment for some but not all

docker/Dockerfile

sbc100 · 2022-12-20T11:35:32Z

docker/Dockerfile.dockerignore

@@ -0,0 +1 @@
+# Overrides top level .dockerignore to allow adding files to the image


Can you explain this? What top level file is being refereed too? Can this file simply be call .dockerignore or does it need the longer name?

It's the .dockerignore file in the root of the repository, which excludes all files since they aren't used in that build. This build does need files added.

The naming scheme is required to be like this for docker to pick it up as an override

Can we instead just remove the top level one.. I don't know why that exists.

Thanks @sbc100 - I've removed both. I suspect one motivation may have been the size of the submodules when initializing docker, but even on a slowish CI machine it seems to be just 4s so seems like an overoptimization

https://github.com/anuraaga/wasi-sdk/actions/runs/3728956980/jobs/6324422776#step:9:174

Can you explain more? Surly docker should not need to read the sub modules directory should it? Doesn't it just include what we tell it to?

Unfortunately no, docker begins by sending the entire context to the daemon - usually this is the current directory, . and is recursive. So to prevent transferring things during this step you need to use dockerignore, the values in ADD statements in the Dockerfile don't influence it.

That seems rather unfortunate! So even if I only add one tiny file to the image the docker create command will be O(size of everything my my CWD). I did not know that.

In that case maybe lets leave the top level .dockerignore but have it only ignore the large directories such as dist.

Alternatively, can we run docker from the subdirectory and ADD ../dist?

Nope, docker won't allow adding files that are't in the context.

In that case maybe lets leave the top level .dockerignore but have it only ignore the large directories such as dist.

Since this image needs dist (to be able to package it), I think it would result in the two dockerignore files again, one ignoring dist at top level and the other not. Just to confirm that's ok?

No, I think we should just have one top level dockerignore, and it should just specify the stuff we don't want (i.e. the submodules).

Thanks @sbc100 - added a top level .dockerignore that excludes /src, the submodules folder.

docker/sdk.cmake

.github/workflows/main.yml

anuraaga

Thanks @sbc100 applied some fixes and clarifications

.github/workflows/main.yml

docker/Dockerfile

anuraaga · 2022-12-20T23:38:56Z

docker/Dockerfile.dockerignore

@@ -0,0 +1 @@
+# Overrides top level .dockerignore to allow adding files to the image


It's the .dockerignore file in the root of the repository, which excludes all files since they aren't used in that build. This build does need files added.

The naming scheme is required to be like this for docker to pick it up as an override

sbc100

lgtm % comment/question.

Not sure if @sunfishcode or @pchickey have more to add?

sunfishcode · 2022-12-21T23:31:36Z

I don't have anything to add here; if people would find this useful, let's do it.

sbc100 · 2022-12-22T02:00:42Z

docker/Dockerfile

+ENV RANLIB llvm-ranlib-${LLVM_VERSION}
+
+ENV CFLAGS --target=wasm32-wasi --sysroot=/wasi-sysroot
+ENV CXXFLAGS --target=wasm32-wasi --sysroot=/wasi-sysroot -fno-exceptions


Given that we don't currently automatically set -fno-exceptions in wasi-sdk.cmake, does it make sense to set it here? Seems odd to diverge from the default.

Ah thanks - I've found the flag helpful when building with wasi-sdk but agree both should be changed at the same time so removed it for now

anuraaga · 2022-12-28T00:59:05Z

Thanks for merging @sbc100! I see the action logs look fine and show a pushed image

https://github.com/WebAssembly/wasi-sdk/actions/runs/3787010175/jobs/6438424135

But the package isn't showing up here

https://github.com/orgs/WebAssembly/packages?repo_name=wasi-sdk

I believe this could be because of a repository or organization setting preventing public packages - would you be able to check on that? Thanks!

The binary linux SDK in version 17 requires a newer glibc than we have available, so we have to use our own patched clang. This follows the approach in: WebAssembly/wasi-sdk#271

sbc100 · 2022-12-28T09:55:47Z

Thanks for merging @sbc100! I see the action logs look fine and show a pushed image

https://github.com/WebAssembly/wasi-sdk/actions/runs/3787010175/jobs/6438424135

But the package isn't showing up here

https://github.com/orgs/WebAssembly/packages?repo_name=wasi-sdk

I believe this could be because of a repository or organization setting preventing public packages - would you be able to check on that? Thanks!

Indeed, it shows up for me there, but is marked as "private". I'll see if I can find the setting to make it public.

sbc100 · 2022-12-28T09:58:07Z

It looks like changing the package visibility settings is an organization-wide permission which I don't have. @dschuff @sunfishcode might be able to do it: https://github.com/orgs/WebAssembly/packages/container/wasi-sdk/settings

The binary linux SDK in version 17 requires a newer glibc than we have available, so we have to use our own patched clang. This follows the approach in: WebAssembly/wasi-sdk#271

abrown · 2023-01-03T18:46:52Z

@anuraaga, once the package permissions are figured out, can you follow this up with a quick note (here or in the project README) on how to use this GitHub-hosted image? I'm familiar with Docker but not with the ghcr.io registry or how this image can be used. I would imagine this to look something like docker run ghcr.io:<something?>? And would we need to append clang on the end of that, e.g.?

anuraaga · 2023-01-05T05:38:19Z

Thanks @abrown! Sent #282 with a README section, there isn't anything special about ghcr.io except the image name happens to start with ghcr.io/, all other Docker knowledge will be usable as-is :)

anuraaga added 6 commits December 16, 2022 16:47

Add docker image that can be used to build projects with wasi-sdk sys…

2b21d9c

…root

Fix dist reading

f87c57c

Update main.yml

11d4073

Use env more

0d0cbb0

Merge branch 'wasi-sdk-docker' of github.com:anuraaga/wasi-sdk into w…

bd52a5d

…asi-sdk-docker

Remove temporary packages

b80bc32

sbc100 reviewed Dec 19, 2022

View reviewed changes

Cleanup

dc88c78

anuraaga commented Dec 20, 2022

View reviewed changes

sbc100 reviewed Dec 20, 2022

View reviewed changes

Cleanup

e128883

anuraaga commented Dec 20, 2022

View reviewed changes

anuraaga added 2 commits December 21, 2022 11:55

Cleanup

682585f

Disable C++ exceptions

de8a07d

sbc100 approved these changes Dec 21, 2022

View reviewed changes

Add .dockerignore that excludes submodule sources

d6e293e

sbc100 reviewed Dec 22, 2022

View reviewed changes

sbc100 approved these changes Dec 22, 2022

View reviewed changes

Update Dockerfile

a6d5074

sbc100 approved these changes Dec 27, 2022

View reviewed changes

sbc100 merged commit 646cac5 into WebAssembly:main Dec 27, 2022

thewtex mentioned this pull request Dec 28, 2022

web-wasi: Bump to WASI-SDK 17 dockcross/dockcross#755

Merged

anuraaga mentioned this pull request Jan 5, 2023

Add README section for docker image #282

Merged

anuraaga mentioned this pull request Sep 27, 2023

Set -fno-exceptions in CXXFLAGS #355

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add docker image that can be used to build projects with wasi-sdk sysroot #271

Add docker image that can be used to build projects with wasi-sdk sysroot #271

anuraaga commented Dec 19, 2022

sbc100 Dec 19, 2022

anuraaga Dec 20, 2022

sbc100 Dec 19, 2022

anuraaga Dec 20, 2022

anuraaga left a comment

anuraaga Dec 20, 2022

anuraaga Dec 20, 2022

sbc100 Dec 20, 2022

anuraaga Dec 20, 2022

sbc100 Dec 20, 2022

anuraaga Dec 21, 2022

sbc100 Dec 21, 2022

sbc100 Dec 22, 2022

sbc100 Dec 22, 2022

anuraaga Dec 22, 2022

sbc100 Dec 22, 2022

anuraaga Dec 22, 2022

anuraaga left a comment

anuraaga Dec 20, 2022

sbc100 left a comment

sunfishcode commented Dec 21, 2022

sbc100 Dec 22, 2022

anuraaga Dec 22, 2022

anuraaga commented Dec 28, 2022

sbc100 commented Dec 28, 2022

sbc100 commented Dec 28, 2022

abrown commented Jan 3, 2023

anuraaga commented Jan 5, 2023

		@@ -0,0 +1 @@
		# Overrides top level .dockerignore to allow adding files to the image

Add docker image that can be used to build projects with wasi-sdk sysroot #271

Add docker image that can be used to build projects with wasi-sdk sysroot #271

Conversation

anuraaga commented Dec 19, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anuraaga left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anuraaga left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sbc100 left a comment

Choose a reason for hiding this comment

sunfishcode commented Dec 21, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anuraaga commented Dec 28, 2022

sbc100 commented Dec 28, 2022

sbc100 commented Dec 28, 2022

abrown commented Jan 3, 2023

anuraaga commented Jan 5, 2023