Consider adding PropertiesDefaultProvider implementation

[remkop]

Consider including the PropertiesDefaultProvider implementation of the IDefaultValueProvider interface in the picocli distribution.

[remkop]

@Fernal73 to follow up on our conversion about user-editable defaulting: PropertiesDefaultProvider is included in picocli 4.1, which was released a few days ago.

https://github.com/remkop/picocli/releases/tag/v4.1.0

[remkop]

@Fernal73 mentions several other tools that provide similar functionality:

• https://prettier.io/docs/en/cli.html
• https://spotbugs.readthedocs.io/en/stable/running.html (the -userPrefs option)

[linusjf]

It's interesting to see how prettier handles config files. There's even more flexibility.

[remkop]

I haven't tested this, but I believe this can be implemented in picocli with multiple passes:

1. parse once, looking only for --no-config and --config-precedence
2. if --no-config was specified, do not install a default provider, and parse the input again;
3. if --config-precedence=cli-override was specified, install PropertiesDefaultProvider and parse the input again;
4. if --config-precedence=file-override was specified, do not install a default provider, but read the config file as an @-file, add the path to this @-file to the command line arguments and parse the input again;
5. if --config-precedence=prefer-file was specified, install PropertiesDefaultProvider and call CommandLine.execute with an empty command line arguments array.

Admittedly, point 4 may not be as simple as that... It may be necessary to inspect the ParseResult and inspect which options were actually specified on the command line, to determine whether to use the value from the command line or from the configuration.

[linusjf]

My understanding of point 4 is that the config file is still a properties file except that its values trump the CLI options. You do bring up an interesting point if you support @files simultaneously. My understanding is that it be treated as an extension of CLI options and used or discarded as per the policy option provided and if respecified (and permitted by policy) , parsed backwards so that the last value specified is the value used.
What is to be done about single-valued params in @files? If treated as command-line expansions, they shouldn't be allowed to be on the command line twice or more including @files. What about toggle parameters, though? They can be specified multiple times.

[linusjf]

How would argument groups resolve this?

[remkop]

There is an option to allow overwriting single-value options, and by default picocli does not toggle booleans when they are specified multiple times.

However, as I thought some more about this, I don't think the @-file approach is going to work. For one thing, the current PropertiesDefaultProvider expects the file to have key-value pairs like option=value, not --option=value.

That could be fixed, of course, but another (bigger) problem is multi-value options; if the user specifies a few values, and the config file also has one or more values, then the collection or map would end up with the union of both, instead of only the config file values.

It may be easier to implement --config-precedence=file-override with something like this:

1. create a CommandLine instance, install PropertiesDefaultProvider and call CommandLine.parseArgs with an empty command line arguments array. Let's call this the config instance.
2. create another, separate, CommandLine instance without a default provider (let's call it the userInput instance), and parse the command line input with parseArgs; this returns a ParseResult object
3. Loop over the matched options and matched positional parameters in the ParseResult from the userInput instance that were matched on the command line, and for each of these, check if the same option or positional parameter in the config instance has a default value. If not, overwrite the value with the userInput value.
4. Invoke the business logic on the config instance.

This would also work for argument groups, although it is more cumbersome to loop over those from the ParseResult.

[linusjf]

I'm inclined to agree that having properties and @files makes it overtly complex for the command-line user. Implementation-wise, if you treat the @file as command line expansion, it's pretty much the same. I was wondering if the three options provided by Pretty Printer can be part of the API, probably in the Command Line class. You'd still have multiple passes but it would be simpler.

[linusjf]

Unfortunately, I had to put aside Picocli for a while to do other stuff. I haven't come that far along into the documentation. I was trying out the samples sequentially while working through it.

[remkop]

@Fernal73 If there is a feature you want added to picocli, it may be better to raise a new ticket so I won't forget about it. :-)

That said, in the spirit of "make common things easy and uncommon things possible", I believe it made sense to add PropertiesDefaultProvider to the library, but more complex use cases like configurations that override command-line args should probably be handled in the application.

If you have a concrete use case, I would be happy to work through it together to see how we could make it work. It might make sense to then post the resulting program as an example to picocli-examples to help other users.

[linusjf]

I'm inclined to agree with you. That was my initial reaction when I first discovered Pretty Printer as well. It still requires two passes. I , somehow, got the impression that it'd take four passes. Maybe, I wasn't thinking too clearly.
It depends on how many command-line implementations do something similar. If it's uncommon, it's best to let the application developer do it. If it's increasingly common, then it becomes expected behaviour. Then, the API might need to provide that.

[remkop]

This topic may also be a good idea for an article or blog post.

I added Advanced Configuration to my list of ideas for articles.

[linusjf]

My experience with command-line tools is limited. However, the Prettier CLI---ignoring CLI options---is sufficiently similar to specifying a properties file last in SpotBugs. That would merit investigating allowing an API setting that has the tool ignore CLI overriding of parameters stored in a properties object. Would it be a tri-state flag with another value to ignore all CLI options? (We're not concerned with positional parameters.)

[remkop]

If there is a specific use case you want to investigate together, would you mind creating a separate ticket that outlines what you want to achieve? (We can decide whether this should be added to the picocli API or not once we have arrived at a solution.)

[linusjf]

I don't have a specific use case at the moment. My suggestions are not easily implementable, in my opinion, unless the property file descriptors are annotated parameters. That's why you need two passes at this point. I'm satisfied with the examples listed as prior art in the documentation. I'll share any more interesting ones should I encounter them.