Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Hacky fix to make mac wheels #15019

Merged
merged 4 commits into from
Feb 7, 2023
Merged

Hacky fix to make mac wheels #15019

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
56f710e
Skip testing PyPy wheels
Feb 7, 2023
212a9e0
Hack the name of the wheel so cibw can test it
Feb 7, 2023
ff8c22f
Changelog
Feb 7, 2023
c1ae188
Apply suggestions from code review
Feb 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion .ci/scripts/auditwheel_wrapper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,16 @@ def cpython(wheel_file: str, name: str, version: Version, tag: Tag) -> str:

check_is_abi3_compatible(wheel_file)

abi3_tag = Tag(tag.interpreter, "abi3", tag.platform)
# HACK: it seems that some older versions of pip will consider a wheel marked
# as macosx_11_0 as incompatible with Big Sur. I haven't done the full archaeology
# here; there are some clues in
# https://github.com/pantsbuild/pants/pull/12857
# https://github.com/pypa/pip/issues/9138
# https://github.com/pypa/packaging/pull/319
# Empirically this seems to work.
# I hate Python packaging, and you should too. Guido, why have you forsaken me?
platform = tag.platform.replace("macosx_11_0", "macosx_10_16")
abi3_tag = Tag(tag.interpreter, "abi3", platform)
DMRobertson marked this conversation as resolved.
Show resolved Hide resolved

dirname = os.path.dirname(wheel_file)
new_wheel_file = os.path.join(
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-artifacts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ jobs:
env:
# Skip testing for platforms which various libraries don't have wheels
# for, and so need extra build deps.
CIBW_TEST_SKIP: pp3{7,9}-* *i686* *musl*
CIBW_TEST_SKIP: pp3*-* *i686* *musl*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per the commit message, I saw a CI failure when testing this out where the linux pypy3.8 build failed: https://github.com/matrix-org/synapse/actions/runs/4117573006/jobs/7109038672

I don't see why pypy3.8 gets special treatment here, but I could also just drop this rather than try to sneak it in.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I failed to review commit by commit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Equally, I failed to point it out!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why pypy3.8 gets special treatment here, but I could also just drop this rather than try to sneak it in.

I think the idea was to test a pypy version, mostly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having slept on it, I wonder if that failure was related to the recent cryptography release. Maybe there is no longer a wheel available for that platform?

We built

 Fixed-up wheel written to /tmp/cibuildwheel/repaired_wheel/matrix_synapse-1.77.0rc1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Which presumably relied on there being a cryptography wheel for that system. But the latest release of cryptography has no such wheel:

$ curl -s https://pypi.org/project/cryptography/39.0.1/#files | grep -o "cryptography.*-pp38.*\.whl"
cryptography-39.0.1-pp38-pypy38_pp73-win_amd64.whl
cryptography-39.0.1-pp38-pypy38_pp73-win_amd64.whl
cryptography-39.0.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
cryptography-39.0.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
cryptography-39.0.1-pp38-pypy38_pp73-win_amd64.whl
cryptography-39.0.1-pp38-pypy38_pp73-win_amd64.whl
cryptography-39.0.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
cryptography-39.0.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl

$ curl -s https://pypi.org/project/cryptography/39.0.1/#files | grep -o "cryptography.*-pp38.*linux.*\.whl"

Not sure why. pyca/cryptography#7795 stopped building PyPy3.7 wheels, and I can't see any recent changes in Cryptography's CI.

There do seem to be pypy 3.9 wheels though:

curl -s https://pypi.org/project/cryptography/39.0.1/#files | grep -o "cryptography.*-pp39.*linux.*\.whl"
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_28_aarch64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_28_aarch64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_24_x86_64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_24_x86_64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_28_aarch64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_28_aarch64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_24_x86_64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_24_x86_64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl
cryptography-39.0.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl

So I'll try testing the PyPy 3.9 wheel in CI.

# Fix Rust OOM errors on emulated aarch64: https://github.com/rust-lang/cargo/issues/10583
CARGO_NET_GIT_FETCH_WITH_CLI: true
CIBW_ENVIRONMENT_PASS_LINUX: CARGO_NET_GIT_FETCH_WITH_CLI
Expand Down
1 change: 1 addition & 0 deletions changelog.d/15019.misc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Workaround packaging pain when making OSX wheels.
DMRobertson marked this conversation as resolved.
Show resolved Hide resolved