Support repeating composite arguments (was: Parameter options/modifiers)

[kravemir]

Picocli supports mixing options and parameters: http://picocli.info/#_mixing_options_and_positional_parameters

I would like to know what options have been specified before certain parameter. Consider such command:

print --paper A4 A.pdf --count 3 B.pdf --rotate left C.pdf -- D.pdf --rotate right E.pdf

Where:

• --paper is an global option,
• --count and --rotate are "local" options, or parameter modifiers,
• -- reset local options.

So, that command would evaluate to:

• globals: paper = A4,
• parameter A.pdf: count = default, rotate = default,
• parameter B.pdf: count = 3, rotate = default,
• parameter C.pdf: count = 3, rotate = left,
• parameter D.pdf: count = default, rotate = default,
• parameter E.pdf: count = default, rotate = right.

Is there support for such things in picocli?

[remkop]

Very interesting use case! I haven't considered this before. So essentially you have repeating composite arguments. ( I may actually change the name of the ticket to this.)

The short answer is that picocli, as of version 3.0.0-alpha-6, does not support functionality like this. I I need to think about this for a while.

[remkop]

Brainstorming a bit: one idea is to introduce a new @Composite annotation, that would allow you to define a group of options and positional parameters that need to be recognized together. Something like the below...

@Command(name = "print")
class PrintCommand {
  enum PaperFormat { A1, A2, A3, A4 }

  @Option(names = "--paper")
  PaperFormat format;

  @CompositePositionalParam(index = "0..*")
  Document[] documents;
}

@Composite
class Document {
  @Parameters(index = "0", arity = "1")
  File file;

  @Option(names = {"-c", "--count"})
  int count = 1; // the default

  enum Rotate { left, right, straight }

  @Option(names = "--rotate")
  Rotate rotate = Rotate.straight; // the default
}

[kravemir]

Composite positional parameters is a good idea. I would rather keep naming consistent, and named it @CompositeParameters.

Modifiers idea

I was also thinking about modifiers vs options. Where modifier would apply to all further parameters, and is reset by --, but option is passed only to one parameter following it. Ie. set count only for one document, or for all following documents.

Depends on use case, but it might be good to leave a choice to an application user, instead of application developer or library developer (forced style).

One idea is to introduce modifier_names to @Option:

  @Option(names = {"-c", "--count"}, modifier_names = {"---count"})
  int count = 1; // the default

So, command-line application user can use -c and --count to specify count for one following document, but can also use ---count to specify count for all following documents until reset.

Modifier reset functionality

One idea to implement reset functionality is to use -- to reset everything. But, it might be unwanted, especially with more complex software having many options, which user would usually like to have preserved. So, just to reset one parameter, user would have to either do:

1. -- ---option_a 123 ---option_b 345 ... - lots of repetition,
2. ---option_to_reset DEFAULT - has to memorize default values.

So, it might be useful to have a possibility to specify reset names for an option, ie.:

  @Option(
    names = {"--rotate"},
    modifier_names = {"---rotate"},
    modifier_reset_names = { "---reset-rotate", "--!rotate" }
  )
  Rotate rotate = Rotate.straight; 

Or, it could be specified by a replacement regex rule, which would generate modifier_reset_names, ie:

@Composite(modifiers_maker = new Replace("---([-a-zA-Z]+)", "---reset-$1"))
class Document

Also, it's not only about easier usage, but about aesthetics of commands.

[remkop]

I agree that @CompositeParameters is a better name.

I think that it makes sense for picocli to provide generic support for composite options and positional parameters but some of what you describe, like the dynamic defaulting behavior, will likely have to be implemented in the application.

Currently, the -- (double hyphen) token is hardcoded in picocli to mean that the remainder are positional parameters. There will need to be a parser option to switch that off, so users can define either options named ”--“ or capture this token in a positional parameter.

Then, given input like your original example:

print --paper A4 A.pdf --count 3 B.pdf --rotate left C.pdf -- D.pdf --rotate right E.pdf

I imagine picocli could parse this and provide the application with these Document objects:

Document(file = A.pdf, count = 3,    rotate = null)
Document(file = B.pdf, count = null, rotate = left)
Document(file = C.pdf, count = null, rotate = null)
Document(file = --,    count = null, rotate = null) // reset
Document(file = D.pdf, count = null, rotate = right)
Document(file = E.pdf, count = null, rotate = null)

The application then has enough information to apply its defaulting rules.

Thoughts?

[kravemir]

I think that it makes sense for picocli to provide generic support for composite options and positional parameters

Great. It would be a very useful feature for applications, which take multiple inputs and need to support extra options for individual inputs.

My use-case is different, but very similar. I work on graphical tool, which combines multiple (1 .. n) inputs and produces an output. All these inputs should support individual options.

but some of what you describe, like the dynamic defaulting behavior, will likely have to be implemented in the application.

You're right. Currently, it's bit of over-thinking/over-engineering to have dynamic defaulting behavior. It would be an extra feature, but it's not essential to full-fill use-case: user can repeat arguments, or dynamic defaulting behavior will be implemented in application.

Currently, the -- (double hyphen) token is hardcoded in picocli to mean that the remainder are positional parameters. There will need to be a parser option to switch that off, so users can define either options named ”--“ or capture this token in a positional parameter.

I would rather separate it to an follow-up feature request, as it's an extra, and isn't essential to fulfill use-case. I see, that you've already created #359 for it.

[remkop]

Related: a simpler but similar use case is this groovy command line tool:

grape resolve [-adhisv] (<groupId> <artifactId> <version>)+

This command can accept repeating groups of groupId, artifactId, and version positional parameters. It rejects input where some but not all elements of the group are specified.

[remkop]

Another use case for composite parameters was raised by @bbottema on #434 (resulting in workaround #441). The use case is a console interface for an email client, with options that have many parameters, like this:

--email:from name=String fromAddress=STRING
--email:replyingTo emailMsg=FILE replyToAll=BOOL htmlTemplate=STRING

Question for @bbottema: are all elements in such composite parameters mandatory? Or would you want to support for example omitting the replyToAll=BOOL (in which case the application picks some default)? I'm not sure if the eventual solution will support this, but I'm trying to understand if there is a need for optional elements in composite parameters.

[bbottema]

--email:from name=String fromAddress=STRING

That's the synopsis, but the invocation currently is:

--email:from "some string" "some address string"

In case it keeps working like this, then the answer to your question depends on the implementation of how arguments are referenced after parsing:

If these composite option values would have to be predefined (ie. a class with declared fields), then it would be useful and feasible to have optional fields (being referable by name). However, in my use case I prefer to leave to the composition unbounded (ie. an array of arbitrary number of types of mandatory parameter values). This is because I'm generating the composite signatures in runtime using reflection.

To illustrate, I prefer:

{ String.class, Integer.class, Boolean.class }

over:

class CompositeOptionValue {
   String a;
   Integer b;
   Boolean c;
}

Of course, you could work with a Map here:

"a" : String.class,
"b" : Integer.class,
"c" : Boolean.class

In this case the parsed arguments can be referenced by name but also generated by name in run time, in which case optional values are workable again. Without named arguments, it will become impossible to know which parameters of a java Method should be null.

In general though, I have to add that I don't have many optional parameters, because that would imply that there is a different need than the API provides and I would simply add an overloaded more succinct version with that parameter left out.

[remkop]

Picocli 4.0.0-alpha-1 has been released which includes support for repeating composite groups.
See https://picocli.info/#_argument_groups for details.

Please try this and provide feedback. We can still make changes.

What do you think of the annotations API? What about the programmatic API? Does it work as expected? Are the input validation error messages correct and clear? Is the documentation clear and complete? Anything you want to change or improve? Any other feedback?