--config-settings recognizes only the last option

[ghost]

Description

The doc said:

The user can supply configuration settings using the --config-settings command line option (which can be supplied multiple times, in order to specify multiple settings).

If setting multiple times like this:

pip wheel --config-settings="--build-option=--cffi" --config-settings="--build-option=--avx2" -v .

Only the last option is passed to backend: {'--build-option': '--avx2'}

It seems these code let the last option override the previous options:

pip/src/pip/_internal/cli/cmdoptions.py

Lines 803 to 806 in c987c68

	if dest is None:
	dest = {}
	setattr(parser.values, option.dest, dest)
	dest[key] = val

Expected behavior

All options are passed to backend: {'--build-option': ['--cffi', '--avx2']}

Modify the code like this:

    if dest is None:
        dest = {key: []}
        setattr(parser.values, option.dest, dest)
    dest[key].append(val)

pip version

22.3.1

Python version

3.10

OS

Windows 11

How to Reproduce

1. define a custom PEP-517 hook get_requires_for_build_wheel
2. let the hook print the config_settings argument

    def get_requires_for_build_wheel(config_settings=None):
        print('config_settings argument:', config_settings)
        return []

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[ghost]

@pfmoore the author of the commit cdeb8f9

[ghost]

This command solves the problem.

pip wheel --config-settings="--build-option=--cffi --avx2" -v .

[layday]

I think there's a genuine bug here. Does your workaround work for all backends or does it just happen to work for setuptools because it's splitting (shlex.split) command-line arguments on its side? IOW, how does a command-line user construct a list of config setting values using pip?

[ghost]

I think there's a genuine bug here.

Yes, the behavior is different from the document, reopen the issue.

My workaround works for setuptools backend.

[pfmoore]

IOW, how does a command-line user construct a list of config setting values using pip?

Note the following in the config-settings documentation:

Build frontends SHOULD provide some mechanism for users to specify arbitrary string-key/string-value pairs to be placed in this dictionary.

Specifically, there's no requirement on frontends to provide a mechanism for users to specify anything other than a string value.

The pip example in the PEP is just an example of how pip might implement --config-settings, but it's not actually what we did. I'm not against the idea of allowing something like what is suggested here, but we need to be cautious. It's not pip's responsibility to define the semantics of config_settings, that's for backends, so what should happen is for backend maintainers to agree on what should be allowed as valid config settings, beyond the minimal "string key, string value" case defined by PEP 517. Once there's an agreed "extended standard", then pip can implement it.

For example, what if a user wants to pass a 1-element list to a backend? How would they do that? We can't mandate that backends have to treat a string value the same as a 1-element list containing that value. When I implemented the feature, I toyed with the idea of allowing JSON-encoded non-string values somehow. But supplying such values on the command line rapidly turns into a quoting nightmare, so I backed off and stuck with the minimum required by the standard.

Setuptools currently accepts lists by splitting. Maybe all backends should do that? I don't know, and it's not really my choice. It would be a lot simpler for frontends if that was the decision, of course.

Having said all of the above, I'm not necessarily against someone trying to implement something more complex. But if you do, be aware that any such behaviour will be very much pip-specific, and you can't assume that other frontends (such as build, or conda, or whatever) will behave the same, or indeed will go beyond the string value approach covered by the PEP which pip currently uses.

[layday]

My chief concern is that the homonymous option in build does behave differently. Somebody coming from pip to build, or from build to pip, as I assume has happened here, will find the discrepancy between the two tools surprising. I don't have a preference, but I think we shouldn't diverge without good reason. That could very well mean that build adopts pip's model for parsing config settings.

[pfmoore]

Good point. IMO, sticking with what's standardised is the correct approach here, which probably means string values only for now. If more complex values are important, then I think the standard needs updating to define the minimal set of types frontends should support. For instance, the build API is a Python API, so there's no reason a backend couldn't choose to interpret strings and numbers differently.

[rgommers]

We ran also into this issue in meson-python (xref mesonbuild/meson-python#235) and scipy (xref scipy/scipy#17736 (comment)). cibuildwheel ran into it too (xref pypa/cibuildwheel#1227 (comment)).

pypa/build allows using the same key more than once, and merges the result into a list of strings. It would be very useful for pip and pypa/build to have the same behavior here, and what pypa/build does seems reasonable, and I think also matches the intent of PEP 517 (given the example linked above does match).

Somebody coming from pip to build, or from build to pip, as I assume has happened here, will find the discrepancy between the two tools surprising.

+1 - also timely given the "unification of interfaces would be really nice" discussion happening on Discourse.

If more complex values are important, then I think the standard needs updating to define the minimal set of types frontends should support

I don't think there is anything in PEP 517 about this case either way, aside from the example which explicitly shows that "key1=value1" ... "key1=value2" gets merged into "key1" = ["value1", "value2"] when calling the build backend hook. So I'd suggest following that example. If you want a clarification in writing @pfmoore, where should it go? Writing a whole new PEP just to say "if keys are duplicate, merge them" seems a bit much.

[pfmoore]

Agreed, I'd be happy with a simple PR to change the spec. Unfortunately, it looks like no-one has migrated PEP 517 to https://packaging.python.org/en/latest/specifications/ yet, so it would have to be a PR changing the PEP itself (which is less than ideal, but what we've done in the past so I'm not bothered by it).

A quick post on discourse, asking if any backend would have a problem with frontends treating duplicated keys as a list of values, should be sufficient to confirm consensus. It might also be worth pointing out that this would mean that backends will never receive a single-element list for an option - instead, that would be passed as a non-list string argument. That's the thing I'm most concerned about (a backend saying "this option is always a list", and not expecting this quirk).

To be clear, I'm not expecting any arguments here, I just want to make sure we don't suddenly start sending unexpected config values to backends.

[layday]

What do we want to change in the spec? If it's to provide a schema for config settings, I think that'd be a positive change. But if it's to define how command-line options in frontends should behave, that seems a little excessive. It's not unthinkable that another, unaffiliated frontend might wish to read config settings from JSON, like Paul's described above. Personally, I think it's sufficient for pip and build to decide on a common strategy. If we're unsure whether build's or pip's approach is better, or if we believe there's room for improvement, we can continue on Discourse.

[rgommers]

What do we want to change in the spec?

I think this (part in italics is added): "Build frontends SHOULD provide some mechanism for users to specify arbitrary string-key/string-value pairs to be placed in this dictionary. In case a user provides duplicate string-key's, build frontends SHOULD combine the corresponding string-value's into a list of string-values. For example ..."

Personally, I think it's sufficient for pip and build to decide on a common strategy.

Either way works for me. It seems like a minimal clarification.

[pfmoore]

The one thing that still bugs me is that backends will have to special-case a string value as equivalent to a 1-element list containing that value. I assume setuptools does that, as otherwise the proposed new behaviour wouldn't work at all (and from what I understand it does work in build). But it seems like a weird condition to impose on backends, that they have to allow this.

Given that using a space-separated list of values --config-settings="--build-option=--cffi --avx2" works for setuptools, because the specification of the --build-option` config setting is that it gets split on spaces, do we actually need this change? Are there any backends that require a config option to be supplied as a list? And if so, how do they expect a single-item list to be supplied?

In reality, of course, hardly anyone supports config options, and the setuptools --build-option case is a holdover from the legacy setup.py command line form, so there really isn't any good example of the "right" way to do things here...

[rgommers]

But it seems like a weird condition to impose on backends, that they have to allow this.

Backends do not have to allow this. If they have no use for it, and hence not implement it, then there is also no valid reason for users to provide duplicate keys to the build frontend.

Are there any backends that require a config option to be supplied as a list? And if so, how do they expect a single-item list to be supplied?

Meson-python for one: mesonbuild/meson-python#235. A single-item list works fine with what both pip and build do today.

It's possible I think to have a --config-settings=key="a value", so the backend can't be sure about splitting on spaces. Passing multiple --config-settings seems more unambiguous. And is already done when keys are not the same, probably for the same reason.

[pfmoore]

So meson-python also translates singe strings to 1-item lists here. Cool.

I think the next step is probably for someone to propose a PR to the spec, and a PR for pip.

[eli-schwartz]

Meson (not the PEP 517 build backend) definitely supports receiving config values that contain embedded spaces. In fact, it also supports receiving array types like this: meson setup builddir/ -Dsimpleopt=simpleval "-Doptname=['val1', 'val2', 'val3,has,commas and spaces']" but if you don't have "unusual" values with commas in them, you can just define it as meson setup builddir/ -Dsimpleopt=simpleval -Doptname=val1,val2,val3.

In short, people can put weird things into an option, and having some form of reliably transcribing that is probably useful.

One method is of course --config-settings=json-blob. Another method is having --config-settings=shlex-parsed-string. But they're both quoting hell, just to different degrees, so, I agree -- it's probably best to get out of that game as rapidly as possible and just accumulate opaque values to the right side of an equals sign.

[uranusjr]

If we’re to change the spec, wouldn’t a simpler solution to make config_settings a list of key-value tuples so backends can decide how to parse them?

[rgommers]

@uranusjr that does seem like a nicer design, but it runs the risk of backwards compat issues for some backends I think. If they don't support list of string input, they need to be updated first and then pip would only be able to make that change much later. If it's str | list[str, ...], then there's nothing that backends need to do. Do you think it's worth pushing for that larger change?

[ghost]

IMO we can adopt the above discussion:
1, Only pass the string after the first equal sign to backend. #11681 (comment)
2, Use this form to deal with multiple specifying: str | list[str, ...]. #11681 (comment)

Another question, does the key should be stripped?
If there is a space before the first equal sign, setuptools doesn't recognize this key as --build-option.

pip wheel --config-settings="--build-option = --cffi" -v .

[uranusjr]

If it's str | list[str, ...], then there's nothing that backends need to do

If a user is relying on the current behaviour (a later assignment overwrites the previous value), this would still make the backend crash in a weird way since the backend is only expecting to receive str values. If we want to eliminate this, some sort of flags would be needed in the backend anyway for compatibility. Unless we don’t feel this is worth supporting (by telling users that overwriting values was an accident and considered a bug).

[ghost]

If a user is relying on the current behaviour (a later assignment overwrites the previous value)

This behavior is different from doc, can be seem as a bug.

since the backend is only expecting to receive str values

Backends probably allow the examples in PEP-517.

[pradyunsg]

Unless we don’t feel this is worth supporting (by telling users that overwriting values was an accident and considered a bug)

I don't think it's worth supporting, FWIW.

[rgommers]

A quick post on discourse, asking if any backend would have a problem with frontends treating duplicated keys as a list of values, should be sufficient to confirm consensus.

Done in https://discuss.python.org/t/amending-pep-517-with-a-sentence-on-handling-duplicate-config-settings-keys/23222