Building Swift with Docker

TODO: Update

2017-04-19 Notes:

• a Swift 3.1 build is now compiling successfully
• the Docker build works on both, Raspi or Mac Docker w/ QEmu
  • building on the Mac, in QEmu is much slower
  • building on a Raspi3 takes ~10 hours
• the instructions below are slightly outdated, I moved stuff around

TODO:

• Update instructions below
• TBD: I don't think it actually makes sense to create an image for the build :-) It should be a build script which is run within the regular Ubuntu container.

Building Swift w/ Docker on macOS

I also tried to actually build Swift itself based on Umberto's instructions, but didn't quite get it running yet. Pull requests are very welcome :-)

In this setup I pulled the Swift sources separately on macOS (they are huge) and added some layers to avoid installing them again and again (rpi-swift-302-ctx etc).

The stack:

• rpi-raspbian-swift-build-env is resin/rpi-raspbian w/ the build dependencies
• rpi-raspbian-swift-build302-env just adds in the (unpatched) Swift sources
• rpi-swift-build then tries to build it

How to build Swift via macOS Docker:

First build the Raspbian docker w/ all the tools required to build Swift:

make build-rpi-raspbian-swift-build-env

Next fetch the Swift sources, this takes a LONG time, they are HUGE:

./grab-swift.sh

This results in a tarball swift-checkout-$date.tar.bz2 which contains the swift-checkout directory. It contains a checkout of all the Swift sources, unpatched, HEAD.

So we want to produce another tarball containing a specific Swift version, that is swift-3.0.2-RELEASE or swift-3.1-RELEASE:

./make-swift-branch-tarball.sh swift-checkout swift-3.0.2-RELEASE

That gives you swift-3.0.2-RELEASE.tgz, which you move into rpi-swift-302-ctx. The context directory is uploaded to Docker everytime you build an image, so keep it small ... This one is almost 1GB in size!

The script modifies the swift-checkout directory. If you build another branch tarball, delete swift-checkout and recreate from the swift-checkout-$date.tar.bz2 you got from grab-swift.sh.

Create another Docker image which is helje5/rpi-raspbian-swift-build-env plus just the sources (added to /swiftsrc/):

make build-rpi-raspbian-swift-build302-env

Finally, trigger the actual build:

make build-rpi-swift-build302

And hope. (I'm told that it takes 7 hours on a Raspi2 and 8 DAYS on a Raspi1 :-)

Update 2017-04-13

The build takes about a day in the emulator, I was hoping it would be roughly the same speed (running on a MacPro), but it isn't.

The same process also works when it is triggered in the Raspi, remote controlled from macOS. That build runs about 7 hours on a Raspi3, but eventually breaks with:

--- bootstrap: note: building stage1: /swiftsrc/build/buildbot_linux/llbuild-linux-armv7/bin/swift-build-tool -f /swiftsrc/build/buildbot_linux/swiftpm-linux-armv7/.bootstrap/build.swift-build
*** Error in `/swiftsrc/build/buildbot_linux/llbuild-linux-armv7/bin/swift-build-tool': malloc(): memory corruption: 0x76c3257e ***
--- bootstrap: error: build failed with exit status -6
./swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting

I think this is the state from which uraimo/buildSwiftOnARM creates the tarball we currently use in helje5/rpi-swift / helje5/rpi-swift-dev. At this point pretty much everything is built, including Foundation. But Swift Package Manager cores with the malloc issue.