Universal binary support

[stuartcarnie]

This PR includes all the changes for building iTerm2 as a Universal macOS Binary. There may be edits required so that it can be merged, but most of the time-consuming work as been done. The only change required to iTerm2 itself was to iTermBacktrace.mm (as anticipated).

It includes universal builds for a number of embedded frameworks, including:

• BetterFontPicker.framework
• ColorPicker.framework
• SearchableComboListView.framework
• CoreParse.framework
• Updated NMSSH.framework as a Universal Binary
  • Recompiled OpenSSL's libcrypto.a and libssh.a as fat binaries
  • Recompiled libssh2.a as a fat binary
• Sparkle.framework
• libsixel
  • Embedded the small amount of source and created a static library target to simplify building

[gnachman]

Thank you for taking on this huge project! I think I need to build at least some of these myself so they will have the proper code signature, and also so I know how to do it next time they change. Also, some of this code came from my personal forks (like Sparkle and NMSSH). This was all a bit opaque which I apologize for. I've added submodules with all the external dependencies whose source wasn't already in the repo as of commit e83f5af.

Please help me learn what you did, because I spent some time fighting with libsixel's automake today and it was not much fun.

The parts I'm not sure how to do are:

1. Rebuilding libcrypto and libssh as fat binaries
2. Rebuilding libssh2.a as a fat binary
3. Rebuilding libsixel as a fat binary (and whatever else you had to do, which isn't quite clear to me)

[stuartcarnie]

Hi @gnachman

libsixel

I committed libsixel configured source to the repo, so if you don't mind, you could run with that as-is. I added it as a new static target with multi-architecture support, so it will automatically compile as a fat binary. I've verified it works by decoding multiple images to my terminal using imgcat.

Fat binaries

tl;dr:

1. Compile the target for each architecture (arm64, x86_64) and then use lipo to combine the individual archives into a single, fat archive. To combine multiple libssl architectures, the lipo command will look something like the following:

$ lipo -create -output libssl.a libssl-arm64.a libssl-x86_64.a

And to verify your archive:

$ lipo -info libssl.a

Architectures in the fat file: libssl.a are: x86_64 arm64

OpenSSL

This was the most involved.

1. Checkout OpenSSL_1_1_1-stable branch

Build arm64 target

1. Apply this patch, which adds a darwin-arm64 target to OpenSSL

diff --git Configurations/10-main.conf Configurations/10-main.conf
index 3b07731db8..fdcdb6a514 100644
--- Configurations/10-main.conf
+++ Configurations/10-main.conf
@@ -1557,6 +1557,14 @@ my %targets = (
         bn_ops           => "SIXTY_FOUR_BIT_LONG",
         perlasm_scheme   => "macosx",
     },
+    "darwin64-arm64-cc" => {
+        inherit_from     => [ "darwin-common", asm("aarch64_asm") ],
+        CFLAGS           => add("-Wall"),
+        cflags           => add("-arch arm64"),
+        lib_cppflags     => add("-DL_ENDIAN"),
+        bn_ops           => "SIXTY_FOUR_BIT_LONG",
+        perlasm_scheme   => "ios64",
+    },
 
 ##### GNU Hurd
     "hurd-x86" => {

NOTE: This updated patch ensures arm64 assembly optimizations are utilized, which I missed from my first version.

2. Configure and build arm64 target:

$ ./Configure darwin64-arm64-cc
$ make build_generated
$ make libcrypto.a libssl.a -j4

3. Move the archives to a new name, to save them for later

$ mv libcrypto.a libcrypto-arm64.a
$ mv libssl.a libssl-arm64.a

4. Cleanup

$ make clean

Configure and build the x86_64 target

$ ./Configure darwin64-x86_64-cc
$ make build_generated
$ make libcrypto.a libssl.a -j4

3. Move the archives to a new name

$ mv libcrypto.a libcrypto-x86_64.a
$ mv libssl.a libssl-x86_64.a

Fat binary

1. Make a fat binary of libcrypto:

$ lipo -create -output libcrypto.a libcrypto-arm64.a libcrypto-x86_64.a

2. Make a fat binary of libssl:

$ lipo -create -output libssl.a libssl-arm64.a libssl-x86_64.a

libssh2

I checked out the latest (1.9-DEV), which seems to work fine, but you could fetch a specific version.

Build x86_64 target

1. Generate x86_64 project dir

$ mkdir build_x86_64
$ cd build_x86_64

2. Invoke CMake generator, noting you'll have to replace OPENSSL_ROOT_DIR with your path

$ cmake -DOPENSSL_ROOT_DIR=~/projects/macos/openssl -DBUILD_EXAMPLES=NO -DBUILD_TESTING=NO -DCMAKE_OSX_ARCHITECTURES=x86_64 -DCRYPTO_BACKEND=OpenSSL ..

3. Build target

$ make libssh2 -j4

NOTE: libssh2.a output is in ./src

Build arm64 target

1. Generate x86_64 project dir

$ mkdir build_arm64
$ cd build_arm64

2. Invoke CMake generator, noting you'll have to replace OPENSSL_ROOT_DIR with your path

$ cmake -DOPENSSL_ROOT_DIR=~/projects/macos/openssl -DBUILD_EXAMPLES=NO -DBUILD_TESTING=NO -DCMAKE_OSX_ARCHITECTURES=arm64 -DCRYPTO_BACKEND=OpenSSL ..

3. Build target

$ make libssh2 -j4

Create libssh2 fat binary

1. Change to libssl2 source root

$ cd ..

2. Combine both targets using lipo

$ lipo -create -output libssh2.a build_arm64/src/libssh2.a build_x86_64/src/libssh2.a

[stuartcarnie]

Update: libssh2

I checked out the latest (1.9-DEV), which seems to work fine, but you could fetch a specific version.

Turns out CMake can utilize the ability for the clang toolchain to build multiple architectures in a single compilation using CMAKE_OSX_ARCHITECTURES='arm64;x86_64'

Build fat binary

1. Create a build dir

$ mkdir build
$ cd build

2. Invoke CMake generator, noting you'll have to replace OPENSSL_ROOT_DIR with your path

$ cmake -DOPENSSL_ROOT_DIR=~/projects/macos/openssl -DBUILD_EXAMPLES=NO -DBUILD_TESTING=NO -DCMAKE_OSX_ARCHITECTURES='arm64;x86_64' -DCRYPTO_BACKEND=OpenSSL ..

3. Build target

$ make libssh2 -j4

NOTE: libssh2.a output is in ./src

[stuartcarnie]

I've pushed a new version that is rebased on your latest. This version is using your forks of the NMSSH and Sparkle frameworks.

[tomhamming]

Did you have to modify it further to support older versions of macOS? I used this to build a fat binary of OpenSSL for my Mac app that supports back to 10.10, and initially got warnings about linking against a library built for a newer macOS version than I was targeting. I had to add to the darwin64-x86_64-cc target:

cppflags         => add("-target x86_64-apple-macos10.10")

[gnachman]

I have a makefile that should automate this, but Apple has suggested to me that this might be under NDA for now. Hang tight and I'll try to get a straight answer from them on whether there's any problem making these changes public yet.

@tomhamming You shouldn't need to go back as far as 10.10. iTerm2 only supports 10.12+

[tomhamming]

@gnachman thanks. I was actually building OpenSSL for my app that supports back to 10.10. I mainly asked the question to clarify that that's the proper way to build for older macOS versions - I'm not familiar with make so it took me a while to figure that out. I suspect other developers trying to update their OpenSSL libraries for Apple Silicon will land here after searching, as I did after seeing @stuartcarnie post about this on Twitter.

[gnachman]

Commit 83a3c75 adds Makefile targets to build everything as fat binaries. If you build it on a DTK you should get a fat binary. Please let me know if I missed anything.

Thanks for your help in getting this done, @stuartcarnie. I am allergic to automake and your help was really valuable.

[stuartcarnie]

@gnachman awesome to hear – I'll give master a shot today!

[stuartcarnie]

Works great!