Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix installer.sh to only require /bin/sh. #141

Merged
merged 6 commits into from
Feb 18, 2025
Merged

Fix installer.sh to only require /bin/sh. #141

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5c89f32
Setup docker cmd.
jsirois Feb 17, 2025
72bcea4
Fix up tags for arches with `/` in them.
jsirois Feb 17, 2025
680452c
Fix Windows typing.
jsirois Feb 17, 2025
3cb70af
Actually type:ignore.
jsirois Feb 17, 2025
8c83031
Convert install.sh to Posix sh syntax for wider usability.
jsirois Feb 17, 2025
2d4ab73
Add back ppc64le test skip.
jsirois Feb 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 31 additions & 113 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,28 +39,28 @@ jobs:
include:
- os: ubuntu-24.04
name: Linux x86-64 (musl)
docker-image: python:3.12-alpine
docker-platform: linux/amd64
image: alpine
arch: amd64
- os: ubuntu-24.04
name: Linux x86-64 (glibc)
docker-image: python:3.12-bookworm
docker-platform: linux/amd64
image: debian
arch: amd64
- os: ubuntu-24.04
name: Linux aarch64
docker-image: python:3.12-bookworm
docker-platform: linux/arm64
image: debian
arch: arm64
- os: ubuntu-24.04
name: Linux armv7l
docker-image: python:3.12-bookworm
docker-platform: linux/arm/v7
image: debian
arch: arm/v7
- os: ubuntu-24.04
name: Linux s390x
docker-image: python:3.12-bookworm
docker-platform: linux/s390x
image: debian
arch: s390x
- os: ubuntu-24.04
name: Linux powerpc64le
docker-image: python:3.12-bookworm
docker-platform: linux/ppc64le
image: debian
arch: ppc64le
- os: macos-13
name: macOS x86-64
- os: macos-14
Expand All @@ -74,10 +74,10 @@ jobs:
SCIENCE_AUTH_API_GITHUB_COM_BEARER: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Install the latest version of uv
if: matrix.docker-image == '' && matrix.os != 'windows-arm64'
if: matrix.os != 'windows-arm64'
uses: astral-sh/setup-uv@v5
- name: Setup uv
if: matrix.docker-image == '' && matrix.os != 'windows-arm64'
if: matrix.os != 'windows-arm64'
run: |
export UV="$(which uv)"
"${UV}" -V
Expand Down Expand Up @@ -106,9 +106,6 @@ jobs:
UV_PYTHON_VERSION=cpython-3.12.9-windows-x86_64-none
"${UV}" python install ${UV_PYTHON_VERSION}
echo UV_PYTHON="${UV_PYTHON_VERSION}" >> ${GITHUB_ENV}
- name: Installing emulators
if: matrix.docker-image != ''
run: docker run --privileged --rm tonistiigi/binfmt --install all
- name: Checkout Lift
uses: actions/checkout@v4
- name: Restore MyPy Cache
Expand All @@ -121,32 +118,14 @@ jobs:
key: ${{ matrix.docker-platform || format('{0}-{1}', matrix.os, runner.arch) }}-a-scie-lift-mypy-v1-${{ github.run_id }}
restore-keys: ${{ matrix.docker-platform || format('{0}-{1}', matrix.os, runner.arch) }}-a-scie-lift-mypy-v1
- name: Check Formatting & Lints
if: matrix.docker-image == ''
if: matrix.image == ''
run: |
"${UV}" run dev-cmd ci --skip test
- name: Check Formatting & Lints
if: matrix.docker-image != ''
if: matrix.image != ''
run: |
cat << EOF > _fmt_lint_check.sh
pip install --root-user-action ignore uv
addgroup --gid $(id -g) build
if [ "${{ matrix.docker-image }}" = "python:3.12-alpine" ]; then
adduser -D -g '' -G build -u $(id -u) build
apk add gcc git linux-headers musl-dev python3-dev
else
adduser --disabled-password --gecos '' --gid $(id -g) --uid $(id -u) build
fi

su build -c 'uv run dev-cmd ci --skip test'
EOF

docker run --rm \
-e FORCE_COLOR \
-e SCIENCE_AUTH_API_GITHUB_COM_BEARER \
-v $PWD:/code \
-w /code \
--platform ${{ matrix.docker-platform }} \
${{ matrix.docker-image }} sh -eu _fmt_lint_check.sh
"${UV}" run dev-cmd docker -- --image ${{ matrix.image }} --arch ${{ matrix.arch }} \
ci --skip test
- name: Cache MyPy
uses: actions/cache/save@v4
if: github.ref == 'refs/heads/main'
Expand All @@ -160,98 +139,37 @@ jobs:
echo PYTEST_ADDOPTS="--basetemp C:/tmp/gha/pytest" >> ${GITHUB_ENV}
echo SCIE_BASE=C:/tmp/gha/nce >> ${GITHUB_ENV}
- name: Unit Tests
if: matrix.docker-image == ''
if: matrix.image == ''
run: |
"${UV}" run dev-cmd test -- -vvs
- name: Unit Tests
if: matrix.docker-image != ''
if: matrix.image != ''
run: |
cat << EOF > _test.sh
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this yaml-thrash and the paired intro of scripts/docker/uv.py was just so I could test the install.sh changes locally via uvrc docker -- --image alpine test -- -vvsk test_installer (where uvrc is my alias for uv run dev-cmd). The upshot of streamlined yaml / an easy way to repro CI locally is, of course, generally good.

if [ "${{ matrix.docker-platform }}" = "linux/s390x" ]; then
# This hack gets the PyPy provider tests working on this image. The old PyPy s390x
# distributions dynamically link libffi at an older version than I've been able to
# find a multi-platform image with s390x support for.
ln -s /usr/lib/s390x-linux-gnu/libffi.so.8 /usr/lib/s390x-linux-gnu/libffi.so.6
elif [ "${{ matrix.docker-platform }}" = "linux/ppc64le" ]; then
if [ "${{ matrix.arch }}" = "ppc64le" ]; then
echo "Skipping tests on ppc64le."
exit 0
fi

pip install --root-user-action ignore uv
addgroup --gid $(id -g) build
if [ "${{ matrix.docker-image }}" = "python:3.12-alpine" ]; then
adduser -D -g '' -G build -u $(id -u) build
# N.B.: The bash and curl packages are additional needs for tests. The rest just
# supports building the psutil wheel needed for the science Python distribution.
apk add bash curl gcc git linux-headers musl-dev python3-dev
else
adduser --disabled-password --gecos '' --gid $(id -g) --uid $(id -u) build
fi

su build -c "uv run dev-cmd test -- -vvs"
EOF

docker run --rm \
-e FORCE_COLOR \
-e SCIENCE_AUTH_API_GITHUB_COM_BEARER \
-v $PWD:/code \
-w /code \
--platform ${{ matrix.docker-platform }} \
${{ matrix.docker-image }} sh -eu _test.sh
"${UV}" run dev-cmd docker -- --image ${{ matrix.image }} --arch ${{ matrix.arch }} \
test -- -vvs
- name: Build & Package
if: matrix.docker-image == ''
if: matrix.image == ''
run: |
"${UV}" run dev-cmd package
- name: Build & Package
if: matrix.docker-image != ''
if: matrix.image != ''
run: |
cat << EOF > _package.sh
pip install --root-user-action ignore uv
addgroup --gid $(id -g) build
if [ "${{ matrix.docker-image }}" = "python:3.12-alpine" ]; then
adduser -D -g '' -G build -u $(id -u) build
apk add gcc git linux-headers musl-dev python3-dev
else
adduser --disabled-password --gecos '' --gid $(id -g) --uid $(id -u) build
fi

su build -c 'uv run dev-cmd package'
EOF

docker run --rm \
-e FORCE_COLOR \
-e SCIENCE_AUTH_API_GITHUB_COM_BEARER \
-v $PWD:/code \
-w /code \
--platform ${{ matrix.docker-platform }} \
${{ matrix.docker-image }} sh -eu _package.sh
"${UV}" run dev-cmd docker -- --image ${{ matrix.image }} --arch ${{ matrix.arch }} \
package
- name: Generate Doc Site
if: matrix.docker-image == ''
if: matrix.image == ''
run: |
"${UV}" run dev-cmd doc linkcheck
- name: Generate Doc Site
if: matrix.docker-image != ''
if: matrix.image != ''
run: |
cat << EOF > _doc_linkcheck.sh
pip install --root-user-action ignore uv
addgroup --gid $(id -g) build
if [ "${{ matrix.docker-image }}" = "python:3.12-alpine" ]; then
adduser -D -g '' -G build -u $(id -u) build
apk add gcc git linux-headers musl-dev python3-dev
else
adduser --disabled-password --gecos '' --gid $(id -g) --uid $(id -u) build
fi

su build -c 'uv run dev-cmd doc linkcheck'
EOF

docker run --rm \
-e FORCE_COLOR \
-e SCIENCE_AUTH_API_GITHUB_COM_BEARER \
-v $PWD:/code \
-w /code \
--platform ${{ matrix.docker-platform }} \
${{ matrix.docker-image }} sh -eu _doc_linkcheck.sh
"${UV}" run dev-cmd docker -- --image ${{ matrix.image }} --arch ${{ matrix.arch }} \
doc linkcheck
- name: Cleanup
if: always()
run: |
Expand Down
56 changes: 19 additions & 37 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,28 +56,28 @@ jobs:
include:
- os: ubuntu-24.04
name: Linux x86-64 (musl)
docker-image: python:3.12-alpine
docker-platform: linux/amd64
image: alpine
arch: amd64
- os: ubuntu-24.04
name: Linux x86-64 (glibc)
docker-image: python:3.12-bookworm
docker-platform: linux/amd64
image: debian
arch: amd64
- os: ubuntu-24.04
name: Linux aarch64
docker-image: python:3.12-bookworm
docker-platform: linux/arm64
image: debian
arch: arm64
- os: ubuntu-24.04
name: Linux armv7l
docker-image: python:3.12-bookworm
docker-platform: linux/arm/v7
image: debian
arch: arm/v7
- os: ubuntu-24.04
name: Linux s390x
docker-image: python:3.12-bookworm
docker-platform: linux/s390x
image: debian
arch: s390x
- os: ubuntu-24.04
name: Linux powerpc64le
docker-image: python:3.12-bookworm
docker-platform: linux/ppc64le
image: debian
arch: ppc64le
- os: macos-13
name: macOS x86-64
- os: macos-14
Expand All @@ -97,10 +97,10 @@ jobs:
discussions: write
steps:
- name: Install the latest version of uv
if: matrix.docker-image == '' && matrix.os != 'windows-arm64'
if: matrix.os != 'windows-arm64'
uses: astral-sh/setup-uv@v5
- name: Setup uv
if: matrix.docker-image == '' && matrix.os != 'windows-arm64'
if: matrix.os != 'windows-arm64'
run: |
export UV="$(which uv)"
"${UV}" -V
Expand Down Expand Up @@ -130,39 +130,21 @@ jobs:
"${UV}" python install ${UV_PYTHON_VERSION}
echo UV_PYTHON_ARGS="--python ${UV_PYTHON_VERSION}" >> ${GITHUB_ENV}
- name: Installing emulators
if: matrix.docker-image != ''
if: matrix.image != ''
run: docker run --privileged --rm tonistiigi/binfmt --install all
- name: Checkout lift ${{ needs.determine-tag.outputs.release-tag }}
uses: actions/checkout@v4
with:
ref: ${{ needs.determine-tag.outputs.release-tag }}
- name: Package science ${{ needs.determine-tag.outputs.release-tag }} binary
if: matrix.docker-image == ''
if: matrix.image == ''
run: |
"${UV}" run ${UV_PYTHON_ARGS} dev-cmd package
- name: Package science ${{ needs.determine-tag.outputs.release-tag }} binary
if: matrix.docker-image != ''
if: matrix.image != ''
run: |
cat << EOF > _package.sh
pip install --root-user-action ignore uv
addgroup --gid $(id -g) build
if [ "${{ matrix.docker-image }}" = "python:3.12-alpine" ]; then
adduser -D -g '' -G build -u $(id -u) build
apk add gcc git linux-headers musl-dev python3-dev
else
adduser --disabled-password --gecos '' --gid $(id -g) --uid $(id -u) build
fi

su build -c 'uv run dev-cmd package'
EOF

docker run --rm \
-e FORCE_COLOR \
-e SCIENCE_AUTH_API_GITHUB_COM_BEARER \
-v $PWD:/code \
-w /code \
--platform ${{ matrix.docker-platform }} \
${{ matrix.docker-image }} sh -eu _package.sh
"${UV}" run dev-cmd docker -- --image ${{ matrix.image }} --arch ${{ matrix.arch }} \
package
- name: Generate science ${{ needs.determine-tag.outputs.release-tag }} artifact attestations
uses: actions/attest-build-provenance@v1
with:
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,6 @@ __pycache__/
# Our Sphinx extension (see: docs/_ext/) dynamically generates doc sources here.
/docs/_/

# Various files are copied from the project root to here for building docker images.
/scripts/docker/ephemeral-build-context/

2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ like so:

+ Linux and macOS:
```
curl --proto '=https' --tlsv1.2 -LSsf https://raw.githubusercontent.com/a-scie/lift/main/install.sh | bash
curl --proto '=https' --tlsv1.2 -LSsf https://raw.githubusercontent.com/a-scie/lift/main/install.sh | sh
```
+ Windows PowerShell:
```
Expand Down
Loading
Loading