Skip to content

Commit

Permalink
CI: micro-optimize test collection & pass nox's --no-install
Browse files Browse the repository at this point in the history
Pytest can be pretty slow to collect pip's entire test suite and
prepare for test execution. I've observed a ~15s delay from invoking
pytest to the first test running in CI in the worst case. This can be
improved by reducing how many files pytest has to process while
collecting tests. In short, passing tests/unit is faster than -m unit.

In addition, use nox's --no-install flag to skip redundant build and
install steps on the 2nd nox session invocation (for the integration
tests), which was made possible by the previous commit.
  • Loading branch information
ichard26 committed Dec 25, 2024
1 parent c340d7e commit 5ce1145
Showing 1 changed file with 14 additions and 22 deletions.
36 changes: 14 additions & 22 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -149,17 +149,17 @@ jobs:
- name: Run unit tests
run: >-
nox -s test-${{ matrix.python.key || matrix.python }} --
-m unit
tests/unit
--verbose --numprocesses auto --showlocals
- name: Run integration tests
run: >-
nox -s test-${{ matrix.python.key || matrix.python }} --
-m integration
nox -s test-${{ matrix.python.key || matrix.python }} --no-install --
tests/functional
--verbose --numprocesses auto --showlocals
--durations=5
tests-windows:
name: tests / ${{ matrix.python }} / ${{ matrix.os }} / ${{ matrix.group }}
name: tests / ${{ matrix.python }} / ${{ matrix.os }} / ${{ matrix.group.number }}
runs-on: ${{ matrix.os }}-latest

needs: [packaging, determine-changes]
Expand All @@ -180,7 +180,9 @@ jobs:
# - "3.11"
# - "3.12"
- "3.13"
group: [1, 2]
group:
- { number: 1, pytest-filter: "not test_install" }
- { number: 2, pytest-filter: "test_install" }

steps:
- uses: actions/checkout@v4
Expand All @@ -198,29 +200,19 @@ jobs:
TEMP: "C:\\Temp"

# Main check
- name: Run unit tests
if: matrix.group == 1
- name: Run unit tests (group 1)
if: matrix.group.number == 1
run: >-
nox -s test-${{ matrix.python }} --
-m unit
tests/unit
--verbose --numprocesses auto --showlocals
env:
TEMP: "C:\\Temp"

- name: Run integration tests (group 1)
if: matrix.group == 1
- name: Run integration tests (group ${{ matrix.group.number }})
run: >-
nox -s test-${{ matrix.python }} --
-m integration -k "not test_install"
--verbose --numprocesses auto --showlocals
env:
TEMP: "C:\\Temp"

- name: Run integration tests (group 2)
if: matrix.group == 2
run: >-
nox -s test-${{ matrix.python }} --
-m integration -k "test_install"
nox -s test-${{ matrix.python }} --no-install --
tests/functional -k "${{ matrix.group.pytest-filter }}"
--verbose --numprocesses auto --showlocals
env:
TEMP: "C:\\Temp"
Expand Down Expand Up @@ -251,7 +243,7 @@ jobs:
- name: Run integration tests
run: >-
nox -s test-3.10 --
-m integration
tests/functional
--verbose --numprocesses auto --showlocals
--durations=5
--use-zipapp
Expand Down

0 comments on commit 5ce1145

Please sign in to comment.