Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add arm64 mac and linux wheels #954

Merged
merged 2 commits into from
Sep 27, 2022
Merged

Add arm64 mac and linux wheels #954

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0f98dc0
Add arm64 mac and linux wheels
ddelange Sep 17, 2022
50277d9
Maximize parallellization (#2)
ddelange Sep 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .github/workflows/install-postgres.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,16 @@ if [ "${ID}" = "debian" -o "${ID}" = "ubuntu" ]; then
apt-get install -y --no-install-recommends \
"postgresql-${PGVERSION}" \
"postgresql-contrib-${PGVERSION}"
elif [ "${ID}" = "almalinux" ]; then
yum install -y \
"postgresql-server" \
"postgresql-devel" \
"postgresql-contrib"
ddelange marked this conversation as resolved.
Show resolved Hide resolved
elif [ "${ID}" = "centos" ]; then
el="EL-${VERSION_ID}-$(arch)"
el="EL-${VERSION_ID%.*}-$(arch)"
baseurl="https://download.postgresql.org/pub/repos/yum/reporpms"
yum install -y "${baseurl}/${el}/pgdg-redhat-repo-latest.noarch.rpm"
if [ ${VERSION_ID} -ge 8 ]; then
if [ ${VERSION_ID%.*} -ge 8 ]; then
dnf -qy module disable postgresql
fi
yum install -y \
Expand Down
52 changes: 37 additions & 15 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
submodules: true

- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4

- name: Build source distribution
run: |
Expand All @@ -68,19 +68,35 @@ jobs:
name: dist
path: dist/*.tar.*

build-wheels:
build-wheels-matrix:
needs: validate-release-request
runs-on: ubuntu-latest
outputs:
include: ${{ steps.set-matrix.outputs.include }}
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v4
- run: pip install cibuildwheel==2.10.2
- id: set-matrix
run: |
MATRIX_INCLUDE=$(
{
cibuildwheel --print-build-identifiers --platform linux --arch x86_64,aarch64 | grep cp | jq -Rc '{"only": inputs, "os": "ubuntu-latest"}' \
&& cibuildwheel --print-build-identifiers --platform macos --arch x86_64,arm64 | grep cp | jq -Rc '{"only": inputs, "os": "macos-latest"}' \
&& cibuildwheel --print-build-identifiers --platform windows --arch x86,AMD64 | grep cp | jq -Rc '{"only": inputs, "os": "windows-latest"}'
} | jq -sc
)
echo ::set-output name=include::"$MATRIX_INCLUDE"

build-wheels:
needs: build-wheels-matrix
runs-on: ${{ matrix.os }}
continue-on-error: true
name: Build ${{ matrix.only }}

strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
cibw_python: ["cp36-*", "cp37-*", "cp38-*", "cp39-*", "cp310-*"]
cibw_arch: ["auto64", "auto32"]
exclude:
- os: macos-latest
cibw_arch: "auto32"
- os: ubuntu-latest
cibw_arch: "auto32"
include: ${{ fromJson(needs.build-wheels-matrix.outputs.include) }}

defaults:
run:
Expand All @@ -94,20 +110,26 @@ jobs:
with:
fetch-depth: 50
submodules: true

- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v2

- uses: pypa/cibuildwheel@v2.8.0
- uses: pypa/cibuildwheel@v2.10.2
with:
only: ${{ matrix.only }}
env:
CIBW_BUILD_VERBOSITY: 1
CIBW_BUILD: ${{ matrix.cibw_python }}
CIBW_ARCHS: ${{ matrix.cibw_arch }}
CIBW_MANYLINUX_X86_64_IMAGE: manylinux_2_28
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux_2_28
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could remove these two completely, defaulting to manylinux_2_21 (which is built on centos RHEL7), also wrt backward compatibility ref giampaolo/psutil#2103 (comment)

Though on centos, CI also has postgres installation script issues (I started out with it), something about a bad GPG key also mentioned on the developer's 3rd party yum repo's issue tracker. Thought about doing a yum search (or smth) to download the exact URL for the respective postgres versions on the official repo.

In the end, I happily accepted gg when I saw the green ticks on CI as it is now.

Martin.Obiols.-.Can.t.stop.laughing.at.this.1255206061159714819.mp4

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should try and keep compatibility with 2_21 at least.

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

Copy link
Contributor Author

@ddelange ddelange Sep 27, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I think this thread is irrelevant, the current state of this PR (so building on almalinux8) already produces wheels that are backward compatible all the way to 2_17 ref https://github.com/ddelange/asyncpg/actions/runs/3121931647/jobs/5063553875#step:4:950

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elprans I think in this respect we're good to go 👍


- uses: actions/upload-artifact@v2
with:
name: dist
path: wheelhouse/*.whl

publish-docs:
needs: validate-release-request
needs: [build-sdist, build-wheels]
runs-on: ubuntu-latest

env:
Expand All @@ -121,7 +143,7 @@ jobs:
submodules: true

- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: 3.8

Expand Down
2 changes: 1 addition & 1 deletion tests/test_introspection.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
from asyncpg import connection as apg_con


MAX_RUNTIME = 0.1
MAX_RUNTIME = 0.25


class SlowIntrospectionConnection(apg_con.Connection):
Expand Down