Config override key flag does not work for extend option

[ofek]

Example command:

ruff format --check --diff --config "extend='ruff_defaults.toml'" .

[MichaReiser]

This is an interesting usage of the new override system and should be supported. However, It may not have the desired result depending on your use case:

pyproject.toml

tests\
	tests.config.toml // extends pyproject.toml

Overriding the extend would override the extend value for all configurations, meaning that tests.config.toml would suddenly no longer extend the pyproject.toml. Now, that might be exactly what you want but may be unintentional if the intention was to make all configurations inherit from a base configuration.

CC: @AlexWaygood

[AlexWaygood]

Ah I see -- so, I think this doesn't work because we currently apply the configuration overrides given in the --config flag after any configuration options found in any pyproject.toml/ruff.toml files have been parsed and validated. But this particular override affects which config files ruff should be looking at in the first place, so we'd need to look for it at an earlier stage in the process (I think).

I'm unlikely to have time to look at this in any more depth this week, unfortunately, but I'll work on this as soon as I'm able. Thanks for the report @ofek!

[ofek]

I'm okay with a fix or literally any other way to define an extend via command line! If it means a separate flag then that's cool.

[AlexWaygood]

Yeah, I'll have a think...

If we decide that this kind of thing should be done via a separate flag, then we should probably explicitly reject trying to override extend via --config, on the grounds that it's probably not going to do what you expect, as @MichaReiser mentioned above. So whether we go with "fix behaviour of --config w.r.t. extend" or "introduce another flag", it's likely the behaviour of --config should be changed in some way here :)

[QuentinSoubeyranAqemia]

+1 on this feature
This would be helpful to extend a base configuration with more specific overrides, when the path to the base config is not static and thus cannot be hard-coded into the more-specific configuration.

If that is any indication, when searching the doc I was also expecting a dedicated --extend flag, and tried out the --config override after failing to find one.
Given that --config "KEY = VALUE" has highest priority and thus needs to be applied last, it makes sense to me that --config cannot support --extend behavior as is currently the case.

I suppose a tricky part could be extend chains: with extend in the files, you can only have a linear chain of files extending one another, so the behavior is well-defined. No diamond inheritance.
With both the file passed to --config, and --extend, possibly extending another file, there can be multiple inheritance. This could reasonably result in an error unless a use-case pops up, though?

[zanieb]

Just for some more context, can someone explain to me how --config <path> --config <some-other-override> differs from --config extend=<path> --config <some-other-override>?

[ofek]

Are you implying that specifying multiple configuration files is implicitly an extend operation? If so, then I suppose there is no feature request here but rather a documentation update.

[zanieb]

I'm not sure! The configuration hierarchy is confusing and inheritance via extend is even more complicated. I'm asking for someone to try it, I guess, so we can understand what we're missing.

[ofek]

[zanieb]

Oh okay, I think we can support multiple configuration files on the command line. I originally intended for it to be that way, but I think it might have been cut from scope on the initial implementation. I'd be curious to hear if it would address your use-case to support merging multiple --config file options. Also @AlexWaygood if you have any more context on why we dropped that from scope that'd be helpful, I can't remember if there were reasons it was infeasible.

[QuentinSoubeyranAqemia]

If you can have more than one config file, what happens when the extend-chain is no longer linear?

ruff --config config1.toml --config config2.toml

# config1.toml
extend = "config1_base.toml"

# config2.toml
extend = "config2_base.toml"

I guess this needs to be properly documented.

---

Also, as a workaround for my own use-case, I template the ruff config and dynamically write a temporary file where I replace a placehold with the actual filepath to extend-from, since the CLI itself can't handle it. It's a bit ugly but it works.