Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use uv in Makefile and CI #1546

Merged
merged 41 commits into from
Nov 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
b04128d
update makefile with uv
sydney-runkle Nov 19, 2024
9d3d96c
adding uv to ci
sydney-runkle Nov 19, 2024
73772f9
adding uv.lock
sydney-runkle Nov 19, 2024
00bf032
use sync with venv
sydney-runkle Nov 19, 2024
4451cc5
more attempted fixes
sydney-runkle Nov 19, 2024
7ad9a37
fix os test
sydney-runkle Nov 19, 2024
69a0880
fix ruff
sydney-runkle Nov 19, 2024
fa0ad9e
improved CI
sydney-runkle Nov 19, 2024
fa3457f
fixing linting issue
sydney-runkle Nov 19, 2024
9ea9a23
coverage updates
sydney-runkle Nov 19, 2024
2f57e08
remove unused mypy
sydney-runkle Nov 19, 2024
a80e698
uv run for maturin?
sydney-runkle Nov 19, 2024
a749ef8
uv run
sydney-runkle Nov 19, 2024
36a5885
more uv run updates
sydney-runkle Nov 19, 2024
de7a405
force reinstalls
sydney-runkle Nov 19, 2024
3c3b2e3
deps fix for integration test
sydney-runkle Nov 19, 2024
bf6d59b
try src file
sydney-runkle Nov 19, 2024
bef5e5c
fix linting issues
sydney-runkle Nov 19, 2024
3b7ace0
try project pydantic
sydney-runkle Nov 19, 2024
84eb33b
not cur dir
sydney-runkle Nov 19, 2024
bd62df3
another top level attempt
sydney-runkle Nov 19, 2024
b0f3e62
controlling venv?
sydney-runkle Nov 20, 2024
7b214d2
uv project environment
sydney-runkle Nov 20, 2024
d9238fa
venv
sydney-runkle Nov 20, 2024
5bb7b8b
venv
sydney-runkle Nov 20, 2024
9baefd3
another attempt
sydney-runkle Nov 20, 2024
524edae
another fix
sydney-runkle Nov 20, 2024
ce04ade
make build dev separately
sydney-runkle Nov 20, 2024
fd113cc
debug, I'm confused
sydney-runkle Nov 20, 2024
11beaeb
simplify, try to fix msrv
davidhewitt Nov 20, 2024
e573ee8
fix integration tests, versioned builds
davidhewitt Nov 20, 2024
7c83ed2
next batch of fixes
davidhewitt Nov 20, 2024
9dde9b8
fix codspeed
davidhewitt Nov 20, 2024
da6e6ef
fixup tests and CI
davidhewitt Nov 20, 2024
f822260
fix benchmarks?
davidhewitt Nov 20, 2024
a0e4a1d
debug
davidhewitt Nov 20, 2024
93c2c9d
try install python on codspeed runs
davidhewitt Nov 20, 2024
c360563
fixup arch builds
davidhewitt Nov 20, 2024
9aadc42
simplify, review comments
davidhewitt Nov 20, 2024
3327255
try last fix for arch tests
davidhewitt Nov 21, 2024
d940668
last fix
davidhewitt Nov 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 100 additions & 90 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ on:

env:
COLUMNS: 150
UV_PYTHON: 3.13

jobs:
coverage:
Expand All @@ -30,26 +31,27 @@ jobs:

- run: rustup component add llvm-tools-preview

- name: set up python
uses: actions/setup-python@v5
- name: install uv
uses: astral-sh/setup-uv@v3
with:
python-version: '3.13'
enable-cache: true

- run: pip install -r tests/requirements.txt
- name: install deps
run: uv sync --group testing

- run: rustc --version --verbose

- run: pip install -e .
- run: make build-dev
env:
RUST_BACKTRACE: 1
RUSTFLAGS: '-C instrument-coverage'

- run: pip freeze
- run: uv pip freeze

- run: coverage run -m pytest
- run: uv run coverage run -m pytest

- run: ls -lha
- run: coverage xml
- run: uv run coverage xml

- run: coverage-prepare lcov python/pydantic_core/*.so

Expand Down Expand Up @@ -85,24 +87,27 @@ jobs:
with:
key: test-v3

- name: set up python
uses: actions/setup-python@v5
- name: install uv
uses: astral-sh/setup-uv@v3
with:
python-version: ${{ matrix.python-version }}
allow-prereleases: true
enable-cache: true

- run: pip install -r tests/requirements.txt
- name: install deps
run: uv sync --group testing

- run: pip install -e .
- run: uv pip install -e .
env:
RUST_BACKTRACE: 1

- run: pip freeze
- run: uv pip freeze

- run: pytest
- run: uv run pytest
env:
HYPOTHESIS_PROFILE: slow

env:
UV_PYTHON: ${{ matrix.python-version }}

test-os:
name: test on ${{ matrix.os }}

Expand All @@ -121,20 +126,21 @@ jobs:
- name: cache rust
uses: Swatinem/rust-cache@v2

- name: set up python
uses: actions/setup-python@v5
- name: install uv
uses: astral-sh/setup-uv@v3
with:
python-version: '3.13'
enable-cache: true

- run: pip install -r tests/requirements.txt
- name: install deps
run: uv sync --group testing

- run: pip install -e .
- run: uv pip install -e .
env:
RUST_BACKTRACE: 1

- run: pip freeze
- run: uv pip freeze

- run: pytest
- run: uv run pytest

- run: cargo test

Expand All @@ -145,14 +151,17 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: set up python
uses: actions/setup-python@v5
- name: install uv
uses: astral-sh/setup-uv@v3
with:
python-version: '3.13'
enable-cache: true

- name: install deps
run: uv sync --group testing

- name: resolve MSRV
id: resolve-msrv
run: echo MSRV=`python -c 'import tomllib; print(tomllib.load(open("Cargo.toml", "rb"))["package"]["rust-version"])'` >> $GITHUB_OUTPUT
run: echo MSRV=`uv run python -c 'import tomllib; print(tomllib.load(open("Cargo.toml", "rb"))["package"]["rust-version"])'` >> $GITHUB_OUTPUT

- name: install rust MSRV
uses: dtolnay/rust-toolchain@master
Expand All @@ -162,15 +171,13 @@ jobs:
- name: cache rust
uses: Swatinem/rust-cache@v2

- run: pip install -r tests/requirements.txt

- run: pip install -e .
- run: uv pip install -e .
env:
RUST_BACKTRACE: 1

- run: pip freeze
- run: uv pip freeze

- run: pytest
- run: uv run pytest

- run: cargo test

Expand All @@ -188,22 +195,28 @@ jobs:

steps:
- uses: actions/checkout@v4
- name: set up python
uses: actions/setup-python@v5

- name: install uv
uses: astral-sh/setup-uv@v3
with:
python-version: ${{ matrix.python-version }}
enable-cache: true

- name: install rust stable
uses: dtolnay/rust-toolchain@stable

- name: cache rust
uses: Swatinem/rust-cache@v2

- run: pip install -r tests/requirements.txt
- name: install deps
run: uv sync --group testing

- run: make build-dev

- run: pip freeze
- run: pytest
- run: uv pip freeze
- run: uv run pytest

env:
UV_PYTHON: ${{ matrix.python-version }}

test-pydantic-integration:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -232,20 +245,24 @@ jobs:

- name: install deps
run: |
uv sync --python 3.13 --extra timezone
uv sync --extra timezone
uv pip install maturin pip
davidhewitt marked this conversation as resolved.
Show resolved Hide resolved
uv run bash -c 'cd ../pydantic-core && make build-dev'
uv run bash -c 'cd ../pydantic-core && maturin develop'
working-directory: pydantic

- run: uv --version && uv pip list
working-directory: pydantic
# Run pytest with lax xfail because we often add tests to pydantic
# which xfail on a pending release of pydantic-core

# Run pytest with lax xfail because we often add tests to pydantic
# which xfail on a pending release of pydantic-core
- run: uv run pytest --override-ini=xfail_strict=False
working-directory: pydantic
env:
PYDANTIC_PRIVATE_ALLOW_UNHANDLED_SCHEMA_TYPES: 1

env:
UV_PROJECT_ENVIRONMENT: ${{ github.workspace }}/.venv

lint:
runs-on: ubuntu-latest

Expand All @@ -260,32 +277,21 @@ jobs:
- name: cache rust
uses: Swatinem/rust-cache@v2

- uses: actions/setup-python@v5
with:
python-version: '3.13'

# used to lint js code
- uses: actions/setup-node@v4
with:
node-version: '18'

- uses: actions/cache@v4
id: cache-py
name: cache python
- name: install uv
uses: astral-sh/setup-uv@v3
with:
path: ${{ env.pythonLocation }}
key: >
py
${{ env.pythonLocation }}
${{ hashFiles('tests/requirements-linting.txt') }}
${{ hashFiles('pyproject.toml') }}

- run: pip install -r tests/requirements-linting.txt
if: steps.cache-py.outputs.cache-hit != 'true'

- run: make build-dev
enable-cache: true

- run: pip freeze
- name: install deps
run: |
uv sync --group linting
make build-dev
uv pip freeze

- run: make lint
- run: make pyright
Expand Down Expand Up @@ -319,12 +325,6 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: set up python
id: setup-python
uses: actions/setup-python@v5
with:
python-version: '3.13'

- name: install rust nightly
uses: dtolnay/rust-toolchain@nightly
with:
Expand All @@ -342,7 +342,13 @@ jobs:
version: '3.1.58'
actions-cache-folder: emsdk-cache

- run: pip install 'maturin>=1,<2' 'ruff==0.5.0' typing_extensions
- name: install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true

- name: install deps
run: uv sync --group wasm

- name: build wheels
run: make build-wasm
Expand All @@ -364,6 +370,9 @@ jobs:
name: wasm_wheels
path: dist

env:
UV_PYTHON: 3.12

# https://github.com/marketplace/actions/alls-green#why used for branch protection checks
check:
if: always()
Expand Down Expand Up @@ -533,11 +542,10 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: set up python
uses: actions/setup-python@v5
- name: install uv
uses: astral-sh/setup-uv@v3
with:
python-version: ${{ matrix.interpreter }}
allow-prereleases: true
enable-cache: true

- name: install rust stable
id: rust-toolchain
Expand Down Expand Up @@ -572,10 +580,9 @@ jobs:

- name: generate pgo data
run: |
pip install -U pip
pip install -r tests/requirements.txt
pip install pydantic-core --no-index --no-deps --find-links pgo-wheel --force-reinstall
pytest tests/benchmarks
uv sync --group testing
uv pip install pydantic-core --no-index --no-deps --find-links pgo-wheel --force-reinstall
uv run pytest tests/benchmarks
rustup run ${{ steps.rust-toolchain.outputs.name }} bash -c 'echo LLVM_PROFDATA=$RUSTUP_HOME/toolchains/$RUSTUP_TOOLCHAIN/lib/rustlib/${{ env.RUST_HOST }}/bin/llvm-profdata >> "$GITHUB_ENV"'

- name: merge pgo data
Expand All @@ -601,6 +608,9 @@ jobs:
name: pypi_files_${{ matrix.os }}_${{ matrix.interpreter }}
path: dist

env:
UV_PYTHON: ${{ matrix.interpreter }}

inspect-pypi-assets:
needs: [build, build-sdist, build-pgo]
runs-on: ubuntu-latest
Expand Down Expand Up @@ -667,20 +677,20 @@ jobs:
if command -v apt-get &> /dev/null; then
echo "installing python & pip with apt-get..."
apt-get update
apt-get install -y --no-install-recommends python3 python3-pip python3-venv git
apt-get install -y --no-install-recommends python3 python3-pip python3-venv git curl
else
echo "installing python & pip with apk..."
apk update
apk add python3 py3-pip git
apk add python3 py3-pip git curl
fi
run: |
set -x
python3 -m venv venv
source venv/bin/activate
python3 -m pip install -r tests/requirements.txt
python3 -m pip install pydantic-core --no-index --no-deps --find-links dist --force-reinstall
python3 -m pytest --ignore=tests/test_docstrings.py
python3 -c 'import pydantic_core._pydantic_core; print(pydantic_core._pydantic_core.__version__)'
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
uv sync --frozen --group testing --no-install-project
uv pip install pydantic-core --no-index --no-deps --find-links dist --force-reinstall
uv run --no-sync pytest --ignore=tests/test_docstrings.py
uv run --no-sync python -c 'import pydantic_core._pydantic_core; print(pydantic_core._pydantic_core.__version__)'

test-builds-os:
name: test build on ${{ matrix.os }}
Expand All @@ -695,10 +705,10 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: set up python
uses: actions/setup-python@v5
- name: install uv
uses: astral-sh/setup-uv@v3
with:
python-version: '3.13'
enable-cache: true

- name: get dist artifacts
uses: actions/download-artifact@v4
Expand All @@ -707,9 +717,9 @@ jobs:
merge-multiple: true
path: dist

- run: pip install -r tests/requirements.txt
- run: pip install pydantic-core --no-index --no-deps --find-links dist --force-reinstall
- run: pytest --ignore=tests/test_docstrings.py
- run: uv sync --group testing
- run: uv pip install pydantic-core --no-index --no-deps --find-links dist --force-reinstall
- run: uv run pytest --ignore=tests/test_docstrings.py

release:
needs: [test-builds-arch, test-builds-os, build-sdist, check]
Expand Down
Loading