Use Nix and Nox to get fast local CI

Use shell.nix to bring in all Python versions that we want to support.
Also bring in Nox via its own dependency group.

This, together with noxfile.py defines a set of new "actions" that we
can run locally (and maybe also in CI in a future commit). For now we
define three actions:

  - tests (run on all supported Python versions)
  - format (run formatting actions)
  - lint (run linters)

Examples of how to run this:

  - nox -s tests  # Run test suite on all supported Python versions
  - nox -s tests-3.7  # Run test suite on v3.7 only
  - nox -s format  # Run formatting action (black + isort)
  - nox -s lint  # Run linters (mypy, pylint, isort, black)
  - nox  # Run all of the above

Some complications worth mentioning:

We have organized our dependencies using Poetry dependency groups (see
[1] for more information on those), and we would therefore like to use
those groups when telling Nox which dependencies are needed for each
of the Nox actions described above.

However, we cannot use `poetry install ...` to install these
dependencies, because Poetry insists on installing into its own
virtualenv - i.e. NOT the virtualenv that Nox creates for each session.

We could have used nox-poetry (https://github.com/cjolowicz/nox-poetry),
but unfortunately it does not support Poetry dependency groups, yet[2].

The workaround/solution is to export the required dependency groups from
Poetry into a requirements.txt file that we can then pass on to Nox's
session.install(). This is implemented by the install_groups() helper
function in our noxfile.py.

On my work laptop, the nox command (i.e. running all of the actions)
completes in ~50s for the initial run, and ~17s on subsequent runs
(when Nox can reuse its per-session virtualenvs).

[1]: https://python-poetry.org/docs/master/managing-dependencies/#dependency-groups

[2]: see cjolowicz/nox-poetry#895 or
cjolowicz/nox-poetry#977 for more discussion.

.gitignore

@@ -1,4 +1,4 @@
-*/__pycache__/
-*/*/__pycache__/
+__pycache__/
 dist/
+.nox/
 .vscode/

noxfile.py

@@ -0,0 +1,74 @@
+import hashlib
+from pathlib import Path
+from typing import Iterable
+
+import nox
+
+
+def install_groups(
+    session: nox.Session,
+    *,
+    include: Iterable[str] = (),
+    exclude: Iterable[str] = (),
+    include_self: bool = True,
+) -> None:
+    """Install Poetry dependency groups
+
+    This function installs the given dependency groups into the session's
+    virtual environment. When 'include_self' is true (the default), the
+    function also installs this package (".") and its default dependencies.
+
+    We cannot use `poetry install` directly here, because it ignores the
+    session's virtualenv and installs into Poetry's own virtualenv. Instead, we
+    use `poetry export` with suitable options to generate a requirements.txt
+    file which we can then pass to session.install().
+
+    Auto-skip the `poetry export` step if the poetry.lock file is unchanged
+    since the last time this session was run.
+    """
+    lockdata = Path("poetry.lock").read_bytes()
+    digest = hashlib.blake2b(lockdata).hexdigest()
+    requirements_txt = Path(session.cache_dir, session.name, "reqs_from_poetry.txt")
+    hashfile = requirements_txt.with_suffix(".hash")
+
+    if not hashfile.is_file() or hashfile.read_text() != digest:
+        requirements_txt.parent.mkdir(parents=True, exist_ok=True)
+        argv = [
+            "poetry",
+            "export",
+            "--format=requirements.txt",
+            f"--output={requirements_txt}",
+        ]
+        if include:
+            option = "only" if not include_self else "with"
+            argv.append(f"--{option}={','.join(include)}")
+        if exclude:
+            argv.append(f"--without={','.join(exclude)}")
+        session.run_always(*argv, external=True)
+        hashfile.write_text(digest)
+
+    session.install("-r", str(requirements_txt))
+    if include_self:
+        session.install(".")
+
+
+@nox.session(python=["3.7", "3.8", "3.9", "3.10", "3.11"], reuse_venv=True)
+def tests(session):
+    install_groups(session, include=["test"])
+    session.run("pytest", "-x", "--log-level=debug", *session.posargs)
+
+
+@nox.session(reuse_venv=True)
+def format(session):
+    install_groups(session, include=["dev"], include_self=False)
+    session.run("isort", "fawltydeps", "tests")
+    session.run("black", ".")
+
+
+@nox.session(reuse_venv=True)
+def lint(session):
+    install_groups(session, include=["dev", "test"], include_self=False)
+    session.run("mypy", "fawltydeps", "tests")
+    session.run("pylint", "fawltydeps", "tests")
+    session.run("isort", "fawltydeps", "tests", "--check-only")
+    session.run("black", "--check", ".")

pyproject.toml

@@ -31,6 +31,12 @@ mypy = "^0.991"
 pylint = "^2.15.8"
 types-setuptools = "^65.6.0.2"
 
+[tool.poetry.group.nox]
+optional = true
+
+[tool.poetry.group.nox.dependencies]
+nox = "^2022.11.21"
+
 [tool.black]
 target-version = ["py37"]
 

shell.nix

@@ -8,13 +8,17 @@
 pkgs.mkShell {
   name = "fawltydeps-env";
   buildInputs = with pkgs; [
+    python37
+    python38
+    python39
     (python310.withPackages (pypkgs: with pypkgs; [
       poetry
     ]))
+    python311
   ];
   shellHook = ''
-    poetry env use "$(which python)"
-    poetry install --sync --with=test,dev
+    poetry env use "${pkgs.python310}/bin/python"
+    poetry install --sync --with=nox,test,dev
     source "$(poetry env info --path)/bin/activate"
   '';
 }