Test installing project on CI

[EliahKagan]

This changes the CI workflow to install the project itself to get its dependency, rather than installing the dependency from requirements.txt. This is the more important thing to test, because it verifies that the project is installable and effectively declares the dependencies it needs.

[Byron]

Thanks a lot!

I was never aware the . syntax even existed and wonder if ultimately is just finds requirements.txt by convention, or if there is any other source of dependency information.

[EliahKagan]

The . syntax installs the project defined in the current directory. It doesn't know anything about requirements.txt, but it does know about setup.py, as well as the more modern setup.cfg and pyproject.toml.

It also can handle build backends that are not currently installed. For example, starting in Python 3.12, setuptools is not automatically installed, so in fresh Python 3.12 virtual environments a setup.py script cannot be run directly, but pip install . will still manage to run it. (This is the "Installing backend dependencies" step sometimes shown in pip output.) This works for backends other than setuptools, too. For example, a project that uses poetry and defines everything in pyproject.toml in a way only poetry understands can be installed with pip install . even if poetry is absent; pip installs the poetry build backend.

So setup.py is actually providing the dependencies that pip install . installs for gitdb:

gitdb/setup.py

Line 23 in d22ac20

install_requires=['smmap>=3.0.1,<6'],

That happens to be the same as in requirements.txt, but it doesn't have to be. If it weren't, installing from requirements.txt could make something succeed that would fail when the package is installed from PyPI (hence why I regard pip install . to be a better way to install required dependencies before running tests).

Although pip install . doesn't use requirements.txt, it will still be used if setup.py reads and uses it, as is the case in GitPython.

The other benefit of pip install . is that it installs the project, not just its dependencies. Because this is installing a package--albeit one that it is building on the fly from the working directory--it more closely matches what happens when the project is installed from PyPI. A middle ground can be achieved by also passing -e/--editable, which "installs a package" but the package is just the local files, rather than copies of them. This is very useful for local development, because changes are immediately reflected in the "installed package," but it's not necessary for CI.

(Of course, another way to test behavior of a built sdist or wheel is to actually build one and install from it in a new environment, which tools like tox and nox automate. I do not claim that using pip install . achieves that level of similarity to installing from PyPI.)

[Byron]

Thanks a lot for the explanation, it's much appreciated.

The other benefit of pip install . is that it installs the project, not just its dependencies. Because this is installing a package--albeit one that it is building on the fly from the working directory--it more closely matches what happens when the project is installed from PyPI. A middle ground can be achieved by also passing -e/--editable, which "installs a package" but the package is just the local files, rather than copies of them.

The -e flag seems like the secret ingredient that I have been missing for a long time now. I simply wasn't able to run GitPython locally anymore once the hacks that I added early on (a decade ago?) to allow that were removed.