Migrate from requirements.txt parsing

[astrojuanlu]

It's very typical to see a code block like this in setup.py:

with open("requirements.txt", encoding="utf-8") as f:
    # Make sure we strip all comments and options (e.g "--extra-index-url")
    # that arise from a modified pip.conf file that configure global options
    requires = []
    for line in f:
        req = line.split("#", 1)[0].strip()
        if req and not req.startswith("--"):
            requires.append(req)

setup(
    ...
    install_requires=requires,

However, as pointed out by the setuptools maintainers, requirements.txt files were always meant for pip, not for build backends (hence the stripping of comments and --<options>).

setuptools ships beta support for consuming them in pyproject.toml:

dynamic = ["dependencies"]

...
[tool.setuptools.dynamic]
dependencies = {file = "requirements.txt"}

(see documentation)

But it breaks workflow tools that rely on adding deps statically to [project.dependencies], for example poetry, pdm, rye, uv.

Would you be open to adding an option that helps users migrate from having their deps in requirements.txt to having them in [project.dependencies]?

[diazona]

Ooh, great suggestion, I'm on board with doing this.

But in theory we should already handle the case where the dependencies are manually parsed out of requirements.txt and passed to the install_requires arg, as in your example. What we do in this project is provide a custom setuptools command, and when the setup.py script calls setup(), that creates a setuptools.dist.Distribution object which contains all the arguments passed to setup() and then passes that on to our custom command, and by that point we don't (shouldn't) care whether install_requires came from a hard-coded list or requirements.txt or something else.

So is the ask here to support projects that use tool.setuptools.dynamic.dependencies.file? Or did you just find a project where the computation of dependencies is not working as intended?

[astrojuanlu]

Oh indeed, I tested both with an old-style setup.py parsing the requirements in Python manually as well as a new-style pyproject.toml declaring the dependencies as dynamic, and both produce a fully static pyproject.toml!

I guess the tool is a bit low level still: if there's an existing pyproject.toml, the one produced by uvx setuptools-pyproject-migration doesn't contain any of the other keys, so if you do overwrite it, you miss a lot of information. Not the end of the world - if you are using git, you can inspect the diff and selectively include what you want.

Regardless, I think that's a different issue? I can close this one in the meantime.

[diazona]

Great, thanks for helping confirm that! It's great to get this kind of engagement 🙂

And that's a good point - so far we've been focusing on the case where a project doesn't have a pyproject.toml file at all, but it has always been part of the plan to incorporate the contents of an existing file if it exists. It's probably about time we did that; I'll make it part of the plan for the next release.