[FR] Design and document a stable mechanism to pass `config_settings` to PEP517 backend #3896

mgorny · 2023-04-19T09:13:30Z

Summary

The parameters accepted by the PEP517 build backend don't seem to be documented anywhere. Both --build-option and --global-option are only mentioned in CHANGES.rst, and build_meta.py code is so convoluted it's hard to figure out how they're supposed to be used correctly.

OS / Environment

No response

Additional Information

It would help people use config_settings correctly when building via PEP517 backend, and avoid surprises when incorrect uses stop working.

Code of Conduct

I agree to follow the PSF Code of Conduct

The text was updated successfully, but these errors were encountered:

alex · 2023-04-25T02:45:05Z

As a concrete example of something that I'd benefit from being documented: I was trying to figure out how to pass py-limited-api via --config-settings, and was struggling with the lack of docs on this.

mgorny · 2023-04-25T12:58:45Z

I'd use a way to pass options to build_ext and build_rust commands but I don't think that's possible at all right now.

alex · 2023-04-25T13:42:23Z

That's what the legacy pip wheel --build-options="..." did, however it's a no-op with PEP517 packages (see pypa/pip#6304), and config_settings are supposed to be the right way to do this now.

As you note, since they're not well documented it's a bit hard to piece together how to use them.

abravalheri · 2023-04-25T16:54:20Z

Hello guys, thank you very much for the issue and ping on this topic.

I am afraid that what you guys want to achieve right now is not exactly implemented yet.

I believe people can right now use something like:

python -m build -C--build-option="..."

However, by default the same config_settings will be passed to both wheel and sdist so it is very likely an error will be raised¹.

I have been thinking about this for a while and it is a tricky subject to address...
I think the most likely way forward would be converting the config_settings into a setup.cfg style file in build time and adding it to the list of config files. But that needs some work².

Unless the user specifically calls python -m build --sdist or python -m build --wheel, the options would need to be common for both bdist_wheel and sdist. ↩
I also happens that setuptools processes config files before sys.argv, so it is not a trivial implementation. ↩

alex · 2023-04-25T17:02:01Z

Thanks for the context, after staring (deeply) at various outputs, it seems that passing --config-settings=--build-option=--py-limited-api=cp37 doesn't work because the --py-limited-api=cp37 component is passed to get_requires_for_build_wheel, which doesn't accept it. I'm honestly very unsure if this is a bug.

abravalheri · 2023-04-25T17:24:11Z

It might as well be, I honestly don't know.
Since the initial implementation of the PEP 517 backend the same options config_settings are shared when calling get_requires*, build_sdist and build_wheel¹. I was not involved in this implementation so I don't know exactly what is the reasoning behind this logic.

My opinion is that config_settings needs some work. I am not sure if it is stable enough to be documented for end user consumption. Probably the best the users can do right now is to add/modify the setup.cfg files and add a [bdist_wheel] section there with the relevant configurations.

There was some discussion pointing out that --global-option and --build-option were not compatible with the legacy pip counterparts so I changed the names, however the behaviour of sharing the contents config_settings among the different hooks is more or less preserved. ↩

leofang · 2023-05-15T22:58:30Z

I think #2491 is related.

beckermr · 2023-07-28T10:16:39Z

I think once this removal (pypa/pip#11859) goes through, there won't be a way to pass options through. Certainly syntax like -C--global-option="" does not work currently.

beckermr · 2023-07-28T12:43:38Z

bump @abravalheri - The pip folks don't have any idea about this and so I think there is some urgency to make sure setuptools works with --config-settings.

beckermr · 2023-07-28T12:43:59Z

Agree with @leofang that #2491 is related.

abravalheri · 2023-07-29T12:01:37Z

Right now the only documentation that we are able to provide is something along the lines:

Currently setuptools handles config_settings in the following way:
config_settings={"verbose": "1"} or {"--verbose": None}: increases log verbosity

config_settings={"verbose": "0"} or {"--quiet": None}: reduces log verbosity (there will still be logging, but less then usual).

config_settings={"editable-mode": "strict"} or {"editable-mode": "compat"}, see https://setuptools.pypa.io/en/latest/userguide/development_mode.html.
config_settings={"--global-option": "[...]"}: stopgap solution for users that previously were directly calling the setup.py command in the following way:
python setup.py [...] COMMAND
Note: This is a stopgap solution and likely to change in the future. It is only supported in a best effort fashion and once a better solution is devised support for this way of passing commands will be removed from the code base.
config_settings={"--build-option": "[...]"}: stopgap solution for users that previously were directly calling the setup.py command in the following way:
python setup.py COMMAND [...]
Note 1: During the build the same options will be passed to all COMMAND used by setuptools.build_meta (usually sdist, bdist_wheel, egg_info and dist_info). If one command does not recognise one of the options, you might have an error.
Note 2: This is a stopgap solution and likely to change in the future. It is only supported in a best effort fashion and once a better solution is devised support for this way of passing commands will be removed from the code base.

Personally, I don't think this is satisfactory. That is why I don't think it is worth including in the docs (I don't want to incentivise people thinking this is stable).

To move forward and actually add first class support for config_settings, we need volunteers. I can think of the following tasks before we are able to commit to config_settings:

Create a list of options we want to support.
- I don't think we should support everything the old direct invocation of python setup.py supports, somethings just don't make sense in a world that we have to abide by PEP 517/660. Moreover setuptools.build_meta only invokes sdist, bdist_wheel, dist_info, so it does not make sense to support options for all the commands which are mostly deprecated.
- It is very likely that we would have multiple rounds of discussion once a first version of this list is proposed
Implement a mechanism for passing config_settings into the relevant Python objects used in the build (either the distribution object or the relevant command object).
- Currently this is done via sys.argv. We can keep doing that or we can do something different.
  - The problem with the existing approach in my view is:
    - Command objects usually call other command objects implicitly, e.g. bdist_wheel command may call the egg_info command, and in this case these implicit commands may run with with default options potentially conflicting with config_settings. This is made worse when combined with the fact that each build hook will run in a completely new subprocess.
  - Setting attributes directly into the relevant objects is very challenging because we need to support setup.py files.
    - setup.py files are so unique that the best we can do is to simply execute them. Attempts of parsing it or interrupt the execution before any command run may break the users' assumptions and bespoke imperative code.
    - This creates an isolation layer between setuptools.build_meta and the distribution and command objects, making it difficult for setuptools.build_meta to pass arguments or manipulate attributes directly.

@beckermr or any other person involved in this discussion, would you like to give it a try and propose some PR? It would be nice if we can get help with this topic and that would help to move things forward faster.

beckermr · 2023-07-29T12:12:40Z

I'd be very very happy to make a PR but I don't actually understand what is happening now, what broke, why it broke, and what needs fixing. :(

abravalheri · 2023-07-29T12:23:43Z

There is nothing really broke.
There is only challenges in implementing config_settings as a functionality that would be useful for the users.

What are options that you want to pass via config_settings?

beckermr · 2023-07-29T13:11:21Z

Currently pips global-options passes arbitrary command line switches to the setup.py. This is a common pattern in the wild despite the various PEPs on packaging. I'd like pips config-settings to simply do the same thing. I tried the nested syntax of --config-settings="--global-options=''" and it did not work for me.

Idk what the proper behavior is or if something is buggy.

abravalheri · 2023-07-29T14:36:18Z

Probably this happens because --config-settings="--global-options='...'" is passed to all commands when you invoke pip install . or python -m build. So if there is an option that is specific to bdist_wheel, sdist will also receive it and complain that it does not know that option.

But there is still the problem using sys.argv for passing options into command objects that I described in #3896 (comment).

beckermr · 2023-07-29T15:01:43Z

Oh hmmmm. In my case, the build command didn't see the arguments at all. I'll cool up some examples and dig in a bit.

abravalheri · 2023-07-29T15:11:32Z

Try --build-option in the singular, I think that is how it was named in pip and got implemented in setuptools. See comment in #3896 (comment).

mgorny · 2023-07-29T16:15:26Z

What are options that you want to pass via config_settings?

In Gentoo, we'd really use a way to pass -j to extension building. Right now we need to explicitly call setup.py build_ext -j... to avoid very slow builds on packages with lots of C files.

beckermr · 2023-07-30T10:42:04Z

Here is my very simple setup.py

from setuptools import setup
import sys

print("\n\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
print(sys.argv)
print(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\n")


setup(
    name="blah",
    version="0.1.0",
    author="MRB",
    author_email="MRB@MRB.com",
    packages=["blah"],
)

And here is what I get

$ pip install -vv --config-settings="--build-option=--foo=bar" . 
Using pip 23.2.1 from /Users/beckermr/mambaforge/envs/test-fitsio-install/lib/python3.11/site-packages/pip (python 3.11)
Non-user install because site-packages writeable
Created temporary directory: /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-build-tracker-rcamhhge
Initialized build tracking at /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-build-tracker-rcamhhge
Created build tracker: /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-build-tracker-rcamhhge
Entered build tracker: /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-build-tracker-rcamhhge
Created temporary directory: /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-install-lb65wn8a
Created temporary directory: /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-ephem-wheel-cache-rljl7t_9
Processing /Users/beckermr/Desktop/blah
  Added file:///Users/beckermr/Desktop/blah to build tracker '/private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-build-tracker-rcamhhge'
  DEPRECATION: Config settings are ignored for project file:///Users/beckermr/Desktop/blah. pip 23.3 will enforce this behaviour change. A possible replacement is to use --use-pep517 or add a pyproject.toml file to the project.
  Running setup.py (path:/Users/beckermr/Desktop/blah/setup.py) egg_info for package from file:///Users/beckermr/Desktop/blah
  Created temporary directory: /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-pip-egg-info-1cuxrxr_
  Running command python setup.py egg_info


  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  ['/Users/beckermr/Desktop/blah/setup.py', 'egg_info', '--egg-base', '/private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-pip-egg-info-1cuxrxr_']
  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

  running egg_info
  creating /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-pip-egg-info-1cuxrxr_/blah.egg-info
  writing /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-pip-egg-info-1cuxrxr_/blah.egg-info/PKG-INFO
  writing dependency_links to /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-pip-egg-info-1cuxrxr_/blah.egg-info/dependency_links.txt
  writing top-level names to /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-pip-egg-info-1cuxrxr_/blah.egg-info/top_level.txt
  writing manifest file '/private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-pip-egg-info-1cuxrxr_/blah.egg-info/SOURCES.txt'
  reading manifest file '/private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-pip-egg-info-1cuxrxr_/blah.egg-info/SOURCES.txt'
  writing manifest file '/private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-pip-egg-info-1cuxrxr_/blah.egg-info/SOURCES.txt'
  Preparing metadata (setup.py) ... done
  Source in /Users/beckermr/Desktop/blah has version 0.1.0, which satisfies requirement blah==0.1.0 from file:///Users/beckermr/Desktop/blah
  Removed blah==0.1.0 from file:///Users/beckermr/Desktop/blah from build tracker '/private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-build-tracker-rcamhhge'
Created temporary directory: /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-unpack-09_cmm5d
Building wheels for collected packages: blah
  Created temporary directory: /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-wheel-r17q4kv3
  Building wheel for blah (setup.py) ...   Destination directory: /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-wheel-r17q4kv3
  Running command python setup.py bdist_wheel


  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  ['/Users/beckermr/Desktop/blah/setup.py', 'bdist_wheel', '-d', '/private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-wheel-r17q4kv3']
  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

  running bdist_wheel
  running build
  running build_py
  /Users/beckermr/mambaforge/envs/test-fitsio-install/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
  !!

          ********************************************************************************
          Please avoid running ``setup.py`` directly.
          Instead, use pypa/build, pypa/installer or other
          standards-based tools.

          See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
          ********************************************************************************

  !!
    self.initialize_options()
  installing to build/bdist.macosx-10.9-x86_64/wheel
  running install
  running install_lib
  creating build/bdist.macosx-10.9-x86_64/wheel
  creating build/bdist.macosx-10.9-x86_64/wheel/blah
  copying build/lib/blah/__init__.py -> build/bdist.macosx-10.9-x86_64/wheel/blah
  running install_egg_info
  running egg_info
  writing blah.egg-info/PKG-INFO
  writing dependency_links to blah.egg-info/dependency_links.txt
  writing top-level names to blah.egg-info/top_level.txt
  reading manifest file 'blah.egg-info/SOURCES.txt'
  writing manifest file 'blah.egg-info/SOURCES.txt'
  Copying blah.egg-info to build/bdist.macosx-10.9-x86_64/wheel/blah-0.1.0-py3.11.egg-info
  running install_scripts
  creating build/bdist.macosx-10.9-x86_64/wheel/blah-0.1.0.dist-info/WHEEL
  creating '/private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-wheel-r17q4kv3/blah-0.1.0-py3-none-any.whl' and adding 'build/bdist.macosx-10.9-x86_64/wheel' to it
  adding 'blah/__init__.py'
  adding 'blah-0.1.0.dist-info/METADATA'
  adding 'blah-0.1.0.dist-info/WHEEL'
  adding 'blah-0.1.0.dist-info/top_level.txt'
  adding 'blah-0.1.0.dist-info/RECORD'
  removing build/bdist.macosx-10.9-x86_64/wheel
done
  Created wheel for blah: filename=blah-0.1.0-py3-none-any.whl size=1086 sha256=a7cef5949118017121a2b143169ea2831132c09871a27e7365cc60c56cc3d22d
  Stored in directory: /private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-ephem-wheel-cache-rljl7t_9/wheels/bd/4f/9e/f4b8cde33b8c6ddebba9ed7b2df0ac3fa4df9736a8c38967c8
Successfully built blah
Installing collected packages: blah
  Attempting uninstall: blah
    Found existing installation: blah 0.1.0
    Uninstalling blah-0.1.0:
      Created temporary directory: /Users/beckermr/mambaforge/envs/test-fitsio-install/lib/python3.11/site-packages/~lah-0.1.0.dist-info
      Removing file or directory /Users/beckermr/mambaforge/envs/test-fitsio-install/lib/python3.11/site-packages/blah-0.1.0.dist-info/
      Created temporary directory: /Users/beckermr/mambaforge/envs/test-fitsio-install/lib/python3.11/site-packages/~lah
      Removing file or directory /Users/beckermr/mambaforge/envs/test-fitsio-install/lib/python3.11/site-packages/blah/
      Successfully uninstalled blah-0.1.0

Successfully installed blah-0.1.0
Remote version of pip: 23.2.1
Local version of pip:  23.2.1
Was pip installed by pip? False
Removed build tracker: '/private/var/folders/p6/5q6thssj6sx3drv_mrndv1v40000gn/T/pip-build-tracker-rcamhhge'

I don't see the flags passed to the subcommands. If I add the --use-pep517, then they do get passed, but I don't actually understand why I would need this or what it does.

abravalheri · 2023-07-30T11:22:00Z

I don't see the flags passed to the subcommands. If I add the --use-pep517, then they do get passed, but I don't actually understand why I would need this or what it does.

I see...

@beckermr, this is because pip tries to be conservative (understandably) when "opting into" the new standard¹ and using the build APIs provided by setuptools.

Since your project does not have a pyproject.toml, pip will consider it "a legacy project" and use the "old way of doing things"². In the "old way" there is not such a thing as config_settings (I don't think pip even passes the parameter to setuptools). Please note that so far, this is not related to the setuptools implementation, but rather the heuristics that pip uses to decide which way the project will be built.

When you add a pyproject.toml file to your project with a build-system table (see example in https://setuptools.pypa.io/en/latest/userguide/quickstart.html#basic-use) or when you pass the --use-pep517 flag, then pip will pass config_settings to setuptools.

Once config_settings is passed to setuptools, then you can expect the behaviour I described in #3896 (comment).

By "new standard" I mean the build procedure described by PEP 517 and PEP 660, i.e. the standard that uses setuptools.build_meta APIs. ↩
By "old way of doing things" I mean that pip will call python setup.py sdist or python setup.py bdist_wheel directly instead of using the APIs in setuptools.build_meta for build isolation. ↩

abravalheri · 2023-07-30T11:35:20Z

In Gentoo, we'd really use a way to pass -j to extension building. Right now we need to explicitly call setup.py build_ext -j... to avoid very slow builds on packages with lots of C files.

Currently there is no support for that due to the open challenges I described in #3896 (comment). The following behaviour is relevant to understand the limitations:

setuptools.build_meta (currently) passes options to commands via sys.argv.
build_ext is called implicitly when bdist_wheel is called (so the options in sys.argv do not reach build_ext).

We welcome the community to propose solutions and participate in the implementation that would tackle these challenges. My comment in #3896 (comment) has a few details, but anyone interested in contributing, please feel free to ask questions.

The best workaround available today for people interested in using command options would be add/modify the setup.cfg files with a [COMMAND] section there with the relevant configurations.

mgorny · 2023-07-30T17:59:18Z

I'm afraid I can't add anything that big to my plate.

By the way, I wanted to suggest that perhaps using the same mechanism as used to provide setup.cfg options could help. Unfortunately, editing setup.cfg is non-trivial (i.e. it's not something we can do automatically for all packages), while calling setup.py build_ext ... kinda is (even though it sucks).

eli-schwartz · 2023-09-05T20:50:23Z

Implement a mechanism for passing config_settings into the relevant Python objects used in the build (either the distribution object or the relevant command object).

Currently, options are passed as --flags on sys.argv which are then parsed by a Command object that if I understand correctly expects to own all arguments, fails on unknown ones, and may manually pass known ones on to a subcommand.

Maybe a better approach would be for a global option setup.py --build-options "....." bdist_wheel that is propagated to every subcommand, and contains a dict of command -> options. So you could do something like:

python -m builld -Cbuild-options="{'build_ext': ['j8'], 'bdist_wheel': ['py-limited-api=cp311']}"

A Command object would automatically read the global build-options, extract the key it is interested in, and consider that to be the options to parse instead of sys.argv (on the assumption that people won't both use this method and subcommand-options).

abravalheri · 2023-09-06T14:03:30Z

Hi @eli-schwartz thank you very much for the proposal, I like the way it goes!

Maybe a better approach would be for a global option setup.py --build-options "....." bdist_wheel that is propagated to every subcommand, and contains a dict of command -> options. So you could do something like: ...
A Command object would automatically read the global build-options, extract the key it is interested in, and consider that to be the options to parse instead of sys.argv (on the assumption that people won't both use this method and subcommand-options).

This might be a bit tricky to pull off, considering that we would need to change the way sys.argv is parsed...

I wonder if we can have another alternative, e.g.:

python setup.py --build-options '{"build_ext.parallel": 8, "bdist_wheel.py_limited_api": "cp311"}' bdist_wheel

This might be a bit more straight forward to parse (str.split + setattr(dist.get_command_obj(...), ...)), assuming that the value is either a JSON string or a Python AST literal (which one would be better?)...

Another thing that I was considering is for setuptools.build_meta to create an extra throw-away setup.cfg-file with the given values:

python setup.py --private-autogenerated-cfg-file /tmp/setuptoos.0183293/build.cfg bdist_wheel

Not sure which is better (a JSON/AST-literal string as CLI argument is a bit ugly, but generating a temp file adds a round-trip)...

eli-schwartz · 2023-09-06T14:22:12Z

I'm not sure there's a particular benefit of json over AST literals (in particular; there's no need for parsing it by non-python code), but there is one pretty annoying downside: if you have a bunch of options to set, and are generating this as a multi-line shell code snippet, it's much easier to use trailing commas for each element. This is totally fine in python dicts, but bizarrely was defined as a syntax error for json.

Another thing that I was considering is for setuptools.build_meta to create an extra throw-away setup.cfg-file with the given values:

I think I tried this but could not figure out how to correctly set it, because it protested that there was no such option.

I think it's basically equivalent to the setattr approach? The downside of this is that since it doesn't do real option parsing, you rely on the internal attribute name, which at least means that setting "j8" doesn't work, but I notice you suggested "parallel=8" and that does indeed seem to work.

It will require additional documentation instead of just saying "the canonical way to pass arguments to subcommands", since only some subcommands work...

abravalheri · 2023-09-06T14:44:53Z

I think I tried this but could not figure out how to correctly set it, because it protested that there was no such option.

Yeah, right now it is not implemented, I was just mentioning it as brainstorm, hypothetically. The closest you can do nowadays is:

export DIST_EXTRA_CONFIG=/tmp/setuptools-build.cfg
echo -e '[build_ext]\nparallel = 8\n[bdist_wheel]\npy_limited_api = cp311' > $DIST_EXTRA_CONFIG
python -m build

* with the caveat that some tools (I believe cibuildwheel, but I may be wrong) might already be using $DIST_EXTRA_CONFIG in some circumstances and therefore they might overwrite it.

The idea was to do something similar to it internally in setuptools.build_meta. Define a new internal global option that works as an additional $DIST_EXTRA_CONFIG, translate config_settings into a throw-away cfg file and pass it using this new global option.

you rely on the internal attribute name, which at least means that setting "j8" doesn't work, but I notice you suggested "parallel=8" and that does indeed seem to work.

I based this on

setuptools/setuptools/_distutils/command/build_ext.py

Line 96 in 1ef36f2

('parallel=', 'j', "number of parallel build jobs"),

The CLI parser setuptools uses is inherited from distutils, -j8 is the short-hand version for --parallel=8 (argparse-style). setup.cfg-files accept the long version of the options only.

In terms of implementation I believe that is easier if we bypass the CLI parser... that is why I suggested the simplified <command>.<option> as keys in a dict. This way we don't have to mess around overwriting distutils implementation.

henryiii · 2023-09-14T12:53:29Z

An author-level workaround for this bug is to follow https://setuptools.pypa.io/en/latest/build_meta.html#dynamic-build-dependencies-and-other-build-meta-tweaks (also the recommended way to customize according to the PEP) and add the following wrapper:

# _custom_build/backend.py
from setuptools import build_meta as _orig
from setuptools.build_meta import *


def get_requires_for_build_wheel(config_settings=None):
    return _orig.get_requires_for_build_wheel()


def get_requires_for_build_sdist(config_settings=None):
    return _orig.get_requires_for_build_sdist()


def get_requires_for_build_editable(config_settings=None):
    return _orig. get_requires_for_build_editable()

[build-system]
requires = ["setuptools"]
build-backend = "backend"
backend-path = ["_custom_build"]

Would it make sense to simply update setuptools itself to ignore config_settings in these three hooks? This restores the build<1 behavior, and would allow this bug to be fixed in pip (pypa/pip#12273). Then these could be re-added in setuptools once this is redesigned eventually, but it moves the handling of this setuptools bug into setuptools itself, rather than requiring downstream tooling to not pass anything for these hooks, as other backends assume these hooks actually get config_settings.

eli-schwartz · 2023-09-14T13:58:21Z

An author-level workaround for this bug

I fail to see how authors editing their projects to use a custom build backend that silently ignores all config_settings, is a workaround for the bug that was reported by someone who would like to pass options to setuptools, acknowledges it is currently impossible to do safely, and would like to figure out a solution for being able to achieve the desired goal.

henryiii · 2023-09-14T15:38:46Z

This seems to be the tracking issue in setuptools for the bug that is exposed by fixing the empty config_settings bug in build (fixed in 1.0) and pip (not fixed yet). At least it's the one that's being mentioned currently. The "impossible to do safely" part is helped by disabling setuptools processing --build-option in the get requires hooks if that's what you need.

The actual issue is just "please document how this is done". ¯\_(ツ)_/¯

custom build backend that silently ignores all config_settings

It does not silently ignore all config_settings. It silently ignores all config_settings in hooks that do not support them currently in pip or older build. It's only a fix if you want custom bdist_wheel =options but don't want those options to be passed to egg_info. And it perfectly replicates the build<1 and current pip behavior, since those do not pass anything into these hooks.

abravalheri · 2023-10-02T11:41:48Z

I am renaming this issue because I believe that only documenting config_settings will not do what users are after.

abravalheri · 2023-10-16T09:46:37Z

I opened a discussion in #4083 for a possible design.

mgorny added documentation Needs Triage Issues that need to be evaluated for severity and status. labels Apr 19, 2023

alex mentioned this issue Apr 25, 2023

pip not naming abi3 wheels correctly pypa/pip#6304

Closed

abravalheri added Needs Discussion Issues where the implementation still needs to be discussed. Needs Investigation Issues which are likely in scope but need investigation to figure out the cause labels Apr 25, 2023

amadio mentioned this issue Aug 18, 2023

Impossible to pass CMake options when building only python extension xrootd/xrootd#2062

Closed

mgorny mentioned this issue Aug 22, 2023

how to pass setup.py arguments projg2/gpep517#10

Closed

abravalheri mentioned this issue Sep 6, 2023

Wheel tags pypa/build#202

Closed

This was referenced Sep 14, 2023

_build_in_isolated_env fails to pass in config_settings to builder.get_dependencies pypa/build#264

Closed

get_requires_for_build_wheel hook is invoked without passing config_settings pypa/pip#12273

Open

This was referenced Oct 1, 2023

Deprecate --global-option and --build-option pypa/pip#11859

Open

Setuptools does not pass config_settings through backend #2491

Open

abravalheri changed the title ~~[Docs] config_settings to PEP517 backend are not documented~~ [FR] Design an stable mechanism to pass config_settings to PEP517 backend Oct 2, 2023

abravalheri changed the title ~~[FR] Design an stable mechanism to pass config_settings to PEP517 backend~~ [FR] Design a stable mechanism to pass config_settings to PEP517 backend Oct 2, 2023

abravalheri changed the title ~~[FR] Design a stable mechanism to pass config_settings to PEP517 backend~~ [FR] Design and document a stable mechanism to pass config_settings to PEP517 backend Oct 2, 2023

bryevdv mentioned this issue Oct 17, 2023

New warnings on non-editable build nv-legate/legate#862

Closed

bnavigator mentioned this issue Oct 25, 2023

pip --build-option is deprecated openSUSE/python-rpm-macros#165

Open

nulano mentioned this issue Dec 30, 2023

Fix editable installation with custom build backend and configuration options python-pillow/Pillow#7658

Merged

neworderofjamie mentioned this issue Jan 8, 2024

Step 8 - tidying genn-team/genn#613

Merged

mxmlnkn mentioned this issue Feb 24, 2024

ratarmount v0.14.1 conda-forge/ratarmount-feedstock#4

Closed

5 tasks

gotmax23 mentioned this issue Apr 7, 2024

[BUG] build_meta: prepare_metadata_for_build_wheel hook does not honor --build-option #4293

Closed

sbidoul mentioned this issue Dec 7, 2024

Remove gone_in for issue 11859 (deprecation of --build-option and --global-option) pypa/pip#13106

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FR] Design and document a stable mechanism to pass `config_settings` to PEP517 backend #3896

[FR] Design and document a stable mechanism to pass `config_settings` to PEP517 backend #3896

mgorny commented Apr 19, 2023

alex commented Apr 25, 2023

mgorny commented Apr 25, 2023

alex commented Apr 25, 2023

abravalheri commented Apr 25, 2023 •

edited

Loading

alex commented Apr 25, 2023

abravalheri commented Apr 25, 2023 •

edited

Loading

leofang commented May 15, 2023

beckermr commented Jul 28, 2023

beckermr commented Jul 28, 2023

beckermr commented Jul 28, 2023

abravalheri commented Jul 29, 2023

beckermr commented Jul 29, 2023

abravalheri commented Jul 29, 2023

beckermr commented Jul 29, 2023

abravalheri commented Jul 29, 2023

beckermr commented Jul 29, 2023

abravalheri commented Jul 29, 2023 •

edited

Loading

mgorny commented Jul 29, 2023

beckermr commented Jul 30, 2023

abravalheri commented Jul 30, 2023 •

edited

Loading

abravalheri commented Jul 30, 2023 •

edited

Loading

mgorny commented Jul 30, 2023

eli-schwartz commented Sep 5, 2023 •

edited

Loading

abravalheri commented Sep 6, 2023

eli-schwartz commented Sep 6, 2023

abravalheri commented Sep 6, 2023 •

edited

Loading

henryiii commented Sep 14, 2023

eli-schwartz commented Sep 14, 2023

henryiii commented Sep 14, 2023

abravalheri commented Oct 2, 2023

abravalheri commented Oct 16, 2023

[FR] Design and document a stable mechanism to pass config_settings to PEP517 backend #3896

[FR] Design and document a stable mechanism to pass config_settings to PEP517 backend #3896

Comments

mgorny commented Apr 19, 2023

Summary

OS / Environment

Additional Information

Code of Conduct

alex commented Apr 25, 2023

mgorny commented Apr 25, 2023

alex commented Apr 25, 2023

abravalheri commented Apr 25, 2023 • edited Loading

Footnotes

alex commented Apr 25, 2023

abravalheri commented Apr 25, 2023 • edited Loading

Footnotes

leofang commented May 15, 2023

beckermr commented Jul 28, 2023

beckermr commented Jul 28, 2023

beckermr commented Jul 28, 2023

abravalheri commented Jul 29, 2023

beckermr commented Jul 29, 2023

abravalheri commented Jul 29, 2023

beckermr commented Jul 29, 2023

abravalheri commented Jul 29, 2023

beckermr commented Jul 29, 2023

abravalheri commented Jul 29, 2023 • edited Loading

mgorny commented Jul 29, 2023

beckermr commented Jul 30, 2023

abravalheri commented Jul 30, 2023 • edited Loading

Footnotes

abravalheri commented Jul 30, 2023 • edited Loading

mgorny commented Jul 30, 2023

eli-schwartz commented Sep 5, 2023 • edited Loading

abravalheri commented Sep 6, 2023

eli-schwartz commented Sep 6, 2023

abravalheri commented Sep 6, 2023 • edited Loading

henryiii commented Sep 14, 2023

eli-schwartz commented Sep 14, 2023

henryiii commented Sep 14, 2023

abravalheri commented Oct 2, 2023

abravalheri commented Oct 16, 2023

[FR] Design and document a stable mechanism to pass `config_settings` to PEP517 backend #3896

[FR] Design and document a stable mechanism to pass `config_settings` to PEP517 backend #3896

abravalheri commented Apr 25, 2023 •

edited

Loading

abravalheri commented Apr 25, 2023 •

edited

Loading

abravalheri commented Jul 29, 2023 •

edited

Loading

abravalheri commented Jul 30, 2023 •

edited

Loading

abravalheri commented Jul 30, 2023 •

edited

Loading

eli-schwartz commented Sep 5, 2023 •

edited

Loading

abravalheri commented Sep 6, 2023 •

edited

Loading