[RFC] Support pinning PEP 517 build deps in pyproject.toml

[webknjaz]

What's the problem this feature will solve?

PEP 518 defines a [build-system] section with a required = [] field. But there is no way to tell PEP 517 build front-ends to apply constraints to those build requirements while provisioning an env for them. This makes it hard to go perform predictable builds.

Describe the solution you'd like

I was thinking that pip-tools could facilitate this by either learning to manage that setting in pyproject.toml or by sourcing the requirements from that field to produce a constraints file. The former could allow to keep the builds predictable with any build front-end like pip while the latter would be helpful mostly for more flexible build front-ends like pypa/build that allow the users to provision the envs by themselves.

WDYT folks?

Alternative Solutions

N/A

Additional context

N/A

[Hattshire]

I think it would be great if pip-compile could take the build-system.requires = [] and tool.<name>.metadata.requires = [] fields from the pyproject.toml and generate a requirements.txt file from that, as it does for setup.py and requirements.in

[AndydeCleyre]

But there is no way to tell PEP 517 build front-ends to apply constraints to those build requirements while provisioning an env for them.

Can you please clarify?

You mean the requires field, right?

When I create a package using flit, I'll have something like:

[build-system]
requires = ["flit_core >=2,<4"]
build-backend = "flit_core.buildapi"

PEP 518 specifies that requires is a list of PEP 508 deps, which may look like:

requests [security,tests] >= 2.8.1, == 2.8.* ; python_version < "2.7"

or

pip @ https://github.com/pypa/pip/archive/1.3.1.zip#sha1=da9234ee9982d4bbb3c72346a6de940a148ea686

What kind of constraints application are you talking about that is not demonstrated in these examples?

[webknjaz]

@AndydeCleyre in requires, the requirements listed are usually direct deps having wide ranges of versions. I want an additional lock file. As in pip install -r direct-requires.txt -c comprehensive-pins.txt

[webknjaz]

I've recently clarified that doing PIP_CONSTRAINT=build-constrains.txt python -m build is able to apply constraints to the deps that are listed in [build-system]requires and returned from the dynamic hooks in the chosen PEP 517 backend (setuptools' backend adds a dependency on wheel under the hood, for example).
Done this manually for now: cherrypy/cheroot@5c1ac08.

[gokuatkai]

supporting [build-system.requires] and [project.optional-dependencies] and all the various other dependencies keys would be really great!

[apljungquist]

I too would like to gather requirements from [build-system.requires].

I also want to gather requirements from the deps sections in tox.ini. And gather constraints from openembedded. Support for plugging in gatherers might be neat.

Side note about using constraints files with pip; you may want to prefer PIP_CONSTRAINT over -c because generated pip commands do not respect constraints files in outer install command (yes, I am still annoyed by this 😅).

[apljungquist]

Not sure if the above would be a good way to do it but looks ok to me. Thoughts?