Flag to stop parsing on first unmatched argument

[defnull]

I have a command line utility that discovers subcommands dynamically, based on the parameters given to the main command. In other words: I only know the list of available subcommands after parsing some of the command line parameters and doing some work, e.g. extending and then scanning the classpath.

Something like this: usage: cmd [--plugin-directory=PATH] <plugin-name> [plugin options]. Plugins are discovered dynamically based on the value of the --plugin-directory parameter.

This works (somehow) with picocli because it stops as soon as it finds an unknown parameter. This is exactly the behavior I need.

Currently I parse the command line in two steps: First I build a parser with setUnmatchedArgumentsAllowed(true) and parse everything up to the first unknown parameter. Then, I load config files, extend the classpath, discover all plugins and build a second parser. This time I can add all plugins and let picocli do its job.

With JCommander for example I cannot do this, because it would find plug-in parameters that have the same name as a main parameter and mix them together, regardless of their position, during the first parsing step.

I have some questions:

1. Is the picocli behavior (stopping at the first unknown parameter) intended, documented and can I rely on it, or is it just a coincidence?
2. Can I disable the warning about unused parameters or change the tracer level programmically?
3. Is there a better way?

[remkop]

Interesting use case. I'd like to support it.

Note that picocli's parsing behaviour changed with the picocli 2.0 release to support mixing options with positional parameters on the command line.

As a result, I believe the latest version of picocli now works more similar to JCommander: unmatched commands are added to the list but parsing continues and picocli tries to match options encountered after the unmatched parameter (so it may apply these options to the top-level command).

1. I need to think a little about what would be the best way to support your use case. It could be as simple as a boolean flag stopParsingOnUnmatchedArgument, or it could be something more elaborate.
2. Currently there is no programmatic API to change the trace level, only system properties. There is also no way currently to switch off some warnings but not others. Improvements in this area could be part of the "more elaborate" solution to (1).

Thoughts?

[defnull]

A stopParsingAtFirstUnmatchedArgument feature would be exactly what is needed to support this use case. The important point is that when given the arguments cmd -known -unknown -known the second -known should not be parsed, but remain in the list of unmatched arguments.

[remkop]

I'm still thinking about a good way to handle this in a generic way. A stopParsingAtFirstUnmatchedArgument feature sounds a bit too specific; imagine someone else has a use case for a stopParsingOnSecondUnmatchedArgument feature...

One idea is to have some kind of callback mechanism that would enable users to plug in custom parser behaviour.

Thinking out loud:

• Parser could give callbacks
  1. before processing a command line argument
  2. when (part of an) argument was successfully matched
  3. when (part of an) argument could not be matched
• During the callback, the callback handler needs to decide what to do with the current token. Usually the handler would just tell the parser to "perform the default action", but this is where users could plug in custom behaviour.
• Finally, the return value of the callback lets the callback handler control what to do next. I imagine returning either CONTINUE or ABORT, or throwing an exception. (Other options?)

Thoughts?

[defnull]

Sure, but such an API is hard to get right and boolean flags are easier for the user to understand.

If you add a callback or ParserPolicy API, then the stopParsingAtFirstUnmatchedArgument could still be useful to change the behavior of the default policy.

[remkop]

I just realized that you can accomplish your use case with picocli v2 by parsing the input twice. (This should work even without a stopParsingAtFirstUnmatchedArgument feature.)

I still think that making parsing behaviour configurable will become necessary at some point in the future, but at least this allows you to make progress.

What do you think?

class TopLevelCommand {
    @Option(names = "--plugin-directory", description = "Location of subcommand definitions")
    File pluginDirectory;

    @Option(names = { "-v", "--verbose"}, description = "Global verbosity level")
    boolean verbose = false;

    public static void main(String[] args) {
        TopLevelCommand topLevel = new TopLevelCommand();
        CommandLine firstIteration = new CommandLine(topLevel);
        firstIteration.setUnmatchedArgumentsAllowed(true);
        try {
            firstIteration.parse(args);

            if (topLevel.pluginDirectory == null) {
                // Handle case where no plugin directory was specified.
                // ...
            } else {
                // from the first iteration we only take enough information to load the plugins
                Map<String, Class> subcommands = topLevel.loadPlugins();
                
                TopLevelCommand command = new TopLevelCommand();
                CommandLine secondIteration = new CommandLine(command);

                // register the subcommands
                for (String name : subcommands.keySet()) {
                    secondIteration.addSubcommand(name, subcommands.get(name).newInstance());
                }
                
                // now we have a fully configured parser, let's parse the input again
                List<CommandLine> parseResult = secondIteration.parse(args);

                // handle fully parsed input (with subcommands) 
                // ...
            }
        } catch (ParameterException ex) {
            System.err.println(ex.getMessage);
            ex.getCommandLine().usage(System.err);
        }
    }
}

If pluginDirectory is available you have enough information to load the plugins and all other options can be ignored. The second iteration the other options will be initialized correctly.

[remkop]

I have changed my mind on this and I am planning to support a stopOnUnmatched flag (or similar, name may change).

[remkop]

@defnull Would you mind taking a look at the discussion on #284? I think it relates to your use case and I would be interested in your feedback.

[remkop]

Fixed in master and 2.x branch. Please verify.

[remkop]

Released in 2.3.0