Test Python 3.9 and 3.12 on CI, test minimum dependencies (jupyterlab…

…#1029)

* Test Python 3.9 and 3.13 on CI, test minimum dependencies

* Bump versions in pyproject.toml files, define hatch build

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix spurious dev install in Python unit tests job

* Restore the install from script, add pip list

* Fix the issue

* Satisfy mypy typing, fix missing import

* Fail if any of commands in install script fails

* Revert back the attempt to use 3.13

* Add local dependencies specification

* Update the `requires-python` to `>=3.9,<3.13` in monorepo

* Drop `include_hidden=all_files` from `iglob` instead

as per review users can still do `/learn .*` and
it would be in line how `bash`/`zsh` works (e.g.
`ls * -a` does not files in hidden directories
but `ls .*` does).

* Update Python pin to just `>3.9`

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

.github/workflows/unit-tests.yml

@@ -12,18 +12,32 @@ on:
 
 jobs:
   unit-tests:
-    name: Unit tests
+    name: Unit tests (Python ${{ matrix.python-version }}, ${{ matrix.dependency-type }} dependencies)
     runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - dependency-type: minimum
+            python-version: "3.9"
+          - dependency-type: standard
+            python-version: "3.12"
     steps:
       - name: Checkout
         uses: actions/checkout@v4
 
       - name: Base Setup
         uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
+        with:
+          python_version: ${{ matrix.python-version }}
+          dependency_type: ${{ matrix.dependency-type }}
 
       - name: Install extension dependencies and build the extension
         run: ./scripts/install.sh
 
+      - name: List installed versions
+        run: pip list
+
       - name: Execute unit tests
         run: |
           set -eux

packages/jupyter-ai-magics/jupyter_ai_magics/tests/test_magics.py

@@ -91,8 +91,9 @@ def test_max_history(ip, transcript, max_history, expected_context):
     ai_magics.transcript = transcript.copy()
     ai_magics.max_history = max_history
     provider = ai_magics._get_provider("openrouter")
-    with patch.object(provider, "generate") as generate, patch.dict(
-        os.environ, OPENROUTER_API_KEY="123"
+    with (
+        patch.object(provider, "generate") as generate,
+        patch.dict(os.environ, OPENROUTER_API_KEY="123"),
     ):
         generate.return_value.generations = [[Mock(text="Leet code")]]
         result = ip.run_cell_magic(

packages/jupyter-ai/jupyter_ai/document_loaders/directory.py

@@ -136,7 +136,7 @@ def collect_filepaths(path, all_files: bool):
         filepaths = walk_directory(path, all_files)
     else:
         filepaths = []
-        for glob_path in iglob(str(path), include_hidden=all_files, recursive=True):
+        for glob_path in iglob(str(path), recursive=True):
             if os.path.isfile(glob_path):
                 filepaths.append(Path(glob_path))
     valid_exts = {j.lower() for j in SUPPORTED_EXTS}

pyproject.toml

@@ -5,8 +5,11 @@ build-backend = "hatchling.build"
 [project]
 name = "jupyter_ai_monorepo"
 dynamic = ["version", "description", "authors", "urls", "keywords"]
-requires-python = ">=3.8"
-dependencies = []
+requires-python = ">=3.9"
+dependencies = [
+    "jupyter-ai-magics @ {root:uri}/packages/jupyter-ai-magics",
+    "jupyter-ai @ {root:uri}/packages/jupyter-ai"
+]
 
 [project.optional-dependencies]
 build = []
@@ -22,6 +25,15 @@ text = "BSD 3-Clause License"
 source = "nodejs"
 path = "package.json"
 
+[tool.hatch.build]
+packages = [
+    "packages/jupyter-ai-magics",
+    "packages/jupyter-ai"
+]
+
+[tool.hatch.metadata]
+allow-direct-references = true
+
 [tool.check-manifest]
 ignore = [".*"]
 

scripts/install.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-
+set -eux
 # install core packages
 pip install jupyterlab~=4.0
 cp playground/config.example.py playground/config.py