Test Python 3.9 and 3.12 on CI, test minimum dependencies #1029

krassowski · 2024-10-11T14:47:00Z

fixes /learn with wildcards not working in python 3.10 #1026

Related to multiple past issues where dependencies where misspecified.

for more information, see https://pre-commit.ci

krassowski · 2024-10-11T15:14:32Z

Good, it caught the reported issue:

Bad, the custom install script is reinstalling the pip packages invalidating the resolution in base action. I will remove it.

krassowski · 2024-10-14T13:10:13Z

Looking at the pip list output it does not actually install the minimum required versions in the minimum variant. I think this is likely either:

because the resolution rules do not apply to dependencies (as in metaproject dependencies i.e. jupyter-ai-magics and jupyter-ai) or
because in course of jlpm dev-install the resolutions get overwritten.

This PR could still be merged as-is just to ensure the Python versions are covered; I will iterate on the minimum version installation later either way.

dlqqq

@krassowski Thank you for opening this PR! This is a valuable contribution that will help prevent similar issues from arising in the future. I've left a few points of feedback for you to address below.

Looking at the pip list output it does not actually install the minimum required versions in the minimum variant.

I think to support dependency-type being used in GitHub job runs, we need to figure out how this is being passed to the shell command being run. For example, does this set an environment variable that scripts are supposed to use to determine how to install their dependencies? I can also investigate this myself later if you'd prefer my help on this.

pyproject.toml

packages/jupyter-ai/jupyter_ai/document_loaders/directory.py

krassowski · 2024-10-15T09:04:17Z

To mention it explicitly, the issue with minimum dependency_type is being solved with jupyterlab/maintainer-tools#250.

I think to support dependency-type being used in GitHub job runs, we need to figure out how this is being passed to the shell command being run. For example, does this set an environment variable that scripts are supposed to use to determine how to install their dependencies?

dependency_type is an input to base-setup action (I personally would prefer if it was passed explicitly but I don't feel like trying to fix something that ain't broke). When it is set to minimum three things happen:

pip constraints are discovered from the extension source (how is an implementation detail) and written into a constraints.txt file
the contsraints are installed using pip -c option
the path to the constraints file is exported in PIP_CONSTRAINT environment variable (which is recognised by pip in any subsequent operations).

krassowski · 2024-10-15T10:06:01Z

@dlqqq does jupyterlab/maintainer-tools#250 look good to you?

as per review users can still do `/learn .*` and it would be in line how `bash`/`zsh` works (e.g. `ls * -a` does not files in hidden directories but `ls .*` does).

dlqqq

@krassowski This PR looks great, thank you for contributing this! I've also reviewed jupyterlab/maintainer-tools#250.

…#1029) * Test Python 3.9 and 3.13 on CI, test minimum dependencies * Bump versions in pyproject.toml files, define hatch build * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix spurious dev install in Python unit tests job * Restore the install from script, add pip list * Fix the issue * Satisfy mypy typing, fix missing import * Fail if any of commands in install script fails * Revert back the attempt to use 3.13 * Add local dependencies specification * Update the `requires-python` to `>=3.9,<3.13` in monorepo * Drop `include_hidden=all_files` from `iglob` instead as per review users can still do `/learn .*` and it would be in line how `bash`/`zsh` works (e.g. `ls * -a` does not files in hidden directories but `ls .*` does). * Update Python pin to just `>3.9` --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

krassowski added the maintenance Change related to maintenance of the repository label Oct 11, 2024

Test Python 3.9 and 3.13 on CI, test minimum dependencies

c1e11ba

krassowski force-pushed the test-min-version-and-3.13 branch from 709235e to c1e11ba Compare October 11, 2024 14:49

krassowski and others added 3 commits October 11, 2024 15:56

Bump versions in pyproject.toml files, define hatch build

27f75ed

[pre-commit.ci] auto fixes from pre-commit.com hooks

bbed7ce

for more information, see https://pre-commit.ci

Fix spurious dev install in Python unit tests job

9affd84

krassowski added 4 commits October 11, 2024 16:31

Restore the install from script, add pip list

ea2cf92

Fix the issue

9013e7b

Satisfy mypy typing, fix missing import

a6a8674

Fail if any of commands in install script fails

3d3a037

krassowski changed the title ~~Test Python 3.9 and 3.13 on CI, test minimum dependencies~~ Test Python 3.9 and 3.12 on CI, test minimum dependencies Oct 11, 2024

Revert back the attempt to use 3.13

6080b00

krassowski mentioned this pull request Oct 11, 2024

Update supported Python versions to include 3.13, remove 3.8 #1023

Open

krassowski marked this pull request as ready for review October 11, 2024 15:56

krassowski mentioned this pull request Oct 14, 2024

Constraints file does not get created for monorepos jupyterlab/maintainer-tools#249

Open

Add local dependencies specification

e0fccd7

krassowski mentioned this pull request Oct 14, 2024

Support extracting minimum constraints for monorepo jupyterlab/maintainer-tools#250

Open

dlqqq requested changes Oct 14, 2024

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

pyproject.toml Show resolved Hide resolved

packages/jupyter-ai/jupyter_ai/document_loaders/directory.py Outdated Show resolved Hide resolved

krassowski added 2 commits October 15, 2024 11:13

Update the requires-python to >=3.9,<3.13 in monorepo

9725225

Drop include_hidden=all_files from iglob instead

610eeee

as per review users can still do `/learn .*` and it would be in line how `bash`/`zsh` works (e.g. `ls * -a` does not files in hidden directories but `ls .*` does).

krassowski force-pushed the test-min-version-and-3.13 branch from f435a4d to 610eeee Compare October 15, 2024 10:13

Update Python pin to just >3.9

0f4ea9b

dlqqq approved these changes Oct 15, 2024

View reviewed changes

dlqqq merged commit 41bccfe into jupyterlab:main Oct 15, 2024
10 checks passed

dlqqq mentioned this pull request Oct 15, 2024

/learn with wildcards not working in python 3.10 #1026

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test Python 3.9 and 3.12 on CI, test minimum dependencies #1029

Test Python 3.9 and 3.12 on CI, test minimum dependencies #1029

krassowski commented Oct 11, 2024 •

edited

Loading

krassowski commented Oct 11, 2024

krassowski commented Oct 14, 2024 •

edited

Loading

dlqqq left a comment

krassowski commented Oct 15, 2024

krassowski commented Oct 15, 2024

dlqqq left a comment

Test Python 3.9 and 3.12 on CI, test minimum dependencies #1029

Test Python 3.9 and 3.12 on CI, test minimum dependencies #1029

Conversation

krassowski commented Oct 11, 2024 • edited Loading

krassowski commented Oct 11, 2024

krassowski commented Oct 14, 2024 • edited Loading

dlqqq left a comment

Choose a reason for hiding this comment

krassowski commented Oct 15, 2024

krassowski commented Oct 15, 2024

dlqqq left a comment

Choose a reason for hiding this comment

krassowski commented Oct 11, 2024 •

edited

Loading

krassowski commented Oct 14, 2024 •

edited

Loading