Skip to content

Commit

Permalink
🎬 Build smaller macOS wheels (#54)
Browse files Browse the repository at this point in the history
This PR breaks the universal2 macOS binaries (90+ MB in size) to
architecture-specific wheels. This will provide a smaller download,
since we don't need to ship two binaries.
  • Loading branch information
agriyakhetarpal committed Feb 25, 2024
2 parents a012f73 + 78a3ee9 commit 86bb43e
Show file tree
Hide file tree
Showing 2 changed files with 49 additions and 50 deletions.
45 changes: 2 additions & 43 deletions .github/workflows/cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -305,49 +305,8 @@ jobs:
path: ./wheelhouse/*.whl
if-no-files-found: error

combined_macos_wheels:
name: Combine macOS wheels
runs-on: macos-latest
needs: [macos_x86_64_wheels, macos_arm64_cp38_hotfix, macos_arm64_wheels]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Download macOS wheels for both architectures
uses: actions/download-artifact@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.12

- name: Combine macOS wheels
run: |
mkdir wheelhouse_arm64
mkdir wheelhouse_x86_64
mkdir wheelhouse_universal2
echo "Files in macos_x86_64_wheels:"
ls macos_x86_64_wheels
echo "Files in macos_arm64_wheels:"
ls macos_arm64_wheels
echo "Files in macos_arm64_cp38_hotfix:"
ls macos_arm64_cp38_hotfix
cp macos_x86_64_wheels/*.whl wheelhouse_x86_64/
cp macos_arm64_wheels/*.whl wheelhouse_arm64/
cp macos_arm64_cp38_hotfix/*.whl wheelhouse_arm64/
python -m pip install delocate
python scripts/ci/tools/macos/fuse_wheels.py ./wheelhouse_x86_64 ./wheelhouse_arm64 ./wheelhouse_universal2
- name: Upload combined macOS wheels
uses: actions/upload-artifact@v4
with:
name: combined_macos_wheels
path: ./wheelhouse_universal2/*.whl
if-no-files-found: error

publish:
needs: [sdist, windows_wheels, repair_linux_wheels, combined_macos_wheels]
needs: [sdist, windows_wheels, repair_linux_wheels, macos_arm64_cp38_hotfix, macos_x86_64_wheels, macos_arm64_wheels]
name: Publish to PyPI or TestPyPI
environment: release
permissions:
Expand All @@ -362,7 +321,7 @@ jobs:
- name: Move all artifacts to upload directory
run: |
mkdir upload
mv source_distribution/* windows_wheels/* repaired_linux_wheels/* combined_macos_wheels/* upload/
mv source_distribution/* windows_wheels/* repaired_linux_wheels/* macos_arm64_cp38_hotfix/* macos_x86_64_wheels/* macos_arm64_wheels/* upload/
- uses: pypa/gh-action-pypi-publish@release/v1
if: github.event_name == 'release' && github.event.action == 'published'
Expand Down
54 changes: 47 additions & 7 deletions scripts/ci/tools/macos/fuse_wheels.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,48 @@
#!/usr/bin/env python3
# A wrapper script over delocate-fuse for creating universal2 wheels
# from x86_64 and arm64 wheels, with basic command-line argument parsing

# Usage: fuse_wheels.py <x86_64_wheels_dir> <arm64_wheels_dir> <output_dir>
# Note: these directories must be created beforehand before running this script.
#
# ,---. ,---.
# / /"`.\.--"""--./,'"\ \
# \ \ _ _ / /
# `./ / __ __ \ \,'
# / /_O)_(_O\ \
# | .-' ___ `-. |
# .--| \_/ |--.
# ,' \ \ | / / `.
# / `. `--^--' ,' \
# .-"""""-. `--.___.--' .-"""""-.
# .-----------/ \------------------/ \--------------.
# | .---------\ /----------------- \ /------------. |
# | | `-`--`--' `--'--'-' | |
# | | | |
# | | | |
# | | This script is currently not in use. It is kept for | |
# | | future reference in case we need to ship universal | |
# | | binaries that target both macOS Intel and macOS ARM | |
# | | architectures again. | |
# | | | |
# | | It is to be noted that the issue with universal2 | |
# | | wheels up to Hugo v0.123.3 is not only their size, | |
# | | i.e., 90+ MB owing to ZIP compression rather than | |
# | | TGZ, but also that they contain executables for | |
# | | both architectures rather than a single binary that | |
# | | has been merged with tools like LIPO. | |
# | | | |
# | |_____________________________________________________________| |
# |_________________________________________________________________|
# )__________|__|__________(
# | || |
# |____________||____________|
# ),-----.( ),-----.(
# ,' ==. \ / .== `.
# / ) ( \
# `===========' `===========' hjw
#
#
# A wrapper script over delocate-fuse for creating universal2 wheels
# from x86_64 and arm64 wheels, with basic command-line argument parsing
#
# Usage: fuse_wheels.py <x86_64_wheels_dir> <arm64_wheels_dir> <output_dir>
# Note: these directories must be created beforehand before running this script.

import os
import shutil
Expand Down Expand Up @@ -57,6 +96,7 @@ def copy_if_universal(wheel_name, in_dir, out_dir):
src_path = Path(out_dir) / wheel_name
dest_path = Path(out_dir) / wheel_name_new

print(f"Renamed wheel {wheel_name} to {wheel_name_new}")

wheel_name_string = f"\033[33m{wheel_name}\033[0m"
wheel_name_new_string = f"\033[33m{wheel_name_new}\033[0m"
print(f"✨ Renamed wheel {wheel_name_string} to {wheel_name_new_string} 🎉")
Path(src_path).rename(dest_path)

0 comments on commit 86bb43e

Please sign in to comment.