Picocli 4.6.0

Picocli 4.6.0

The picocli community is pleased to announce picocli 4.6.0.

This release contains new features, bug fixes and other enhancements.

Community Contributions

This release contains many, many community contributions, for which I am extremely grateful. Let's give the contributors some credit!

• Andreas Deininger has been contributing to the documentation and other areas for a long time, but recently went into overdrive :-) and contributed many, many new pull requests to improve the documentation. The user manual and Quick Guide now have a "foldable" table of contents, and examples in tabs, with many additional examples in Kotlin, Scala and Groovy. A lot of work went into this! Many thanks, Andreas!
• Marko Mackic contributed a pull request to add IModelTransformer API for user-defined model transformations after initialization and before parsing.
• Sualeh Fatehi contributed a pull request to the picocli-shell-jline3 module that adds a built-in clear command and improves the help command.
• H.Sakata contributed a pull request that adds support for echo and prompt for interactive options and positional parameters.
• Daniel Gray contributed a bug fix to prevent incorrectly defaulting inherited positional params after a subcommand.
• nveeser-google contributed a fix for compiler warnings about Annotation::getClass and assignment in if condition.
• Petr Hála contributed a pull request to add a section on Mocking to user manual.
• Max Rydahl Andersen contributed a pull request to include jbang in the Build Tools section of the user manual.
• David Phillips contributed a section to the user manual on Really Executable JARs.
• Laurent Almeras contributed a pull request to fix the user manual: @ParentObject should be @ParentCommand.
• Mattias Andersson raised the idea of supporting subcommand methods in Groovy scripts.
• Adrian A. raised the idea of using closures in the picocli annotations in Groovy programs instead of specifying a class.
• Nick Cross raised the idea of inheriting @Command attributes with scope=INHERIT.
• Marko Mackic raised the idea of adding a CommandSpec::removeSubcommand method.
• Max Rydahl Andersen raised the idea of supporting Optional<T> type for options and positional parameters.
• Max Rydahl Andersen and David Walluck raised the idea of supporting key-only Map options (to support -Dkey as well as -Dkey=value).
• David Walluck raised the idea of a "preprocessor" parser plugin.
• Jannick Hemelhof raised the idea of supporting @Spec-annotated members in ArgGroup classes.
• Vitaly Shukela raised a bug report: the error message for unmatched positional argument reports an incorrect index when value equals a previously matched argument.
• drkilikil raised a bug report: MissingParameterException should not be thrown when subcommand has required options and help option is specified on parent command.
• Sebastian Thomschke raised a bug report: ReflectionConfigGenerator should not generate method section in subclass config for private superclass methods in reflect-config.json.
• Lukas Heumos added the picocli-based cli-java template to CookieTemple.
• Sualeh Fatehi raised the idea of adding add CommandLine::getFactory accessor method.
• David Walluck contributed a test improvement that allows the tests to run reliably in more environments.
• Sabrina (link witheld) pointed out various typos in the documentation.

What is in this release

Improved Groovy support: this release introduces a new @PicocliScript2 annotation that adds support for exit codes and @Command-annotated methods to define subcommands. Also, from this release, Groovy programs can use closures in the picocli annotations instead of specifying a class.

From this release, Map options accept key-only parameters, so end users can specify -Dkey as well as -Dkey=value.
There is a new mapFallbackValue attribute that enables this, which can be used to control the value that is put into the map when only a key was specified on the command line.

Also, this release adds support for java.util.Optional<T>: single-value types can be wrapped in an Optional container object when running on Java 8 or higher.
If the option or positional parameter was not specified on the command line, picocli assigns the value Optional.empty() instead of null.

This release also adds support for commands with scope = ScopeType.INHERIT. Commands with this scope have their attributes copied to all subcommands (and sub-subcommands).

New parser plugin: IParameterPreprocessor and new configuration plugin: IModelTransformer.

From this release, @Spec-annotated elements can be used in ArgGroup classes, which can be convenient for validation.

Interactive options and positional parameters can now set echo = true (for non-security sensitive data) so that user input is echoed to the console, and control the prompt text that is shown before asking the user for input.

Help API: this release adds public methods Help.Layout::colorScheme, Help.Layout::textTable, Help.Layout::optionRenderer, Help.Layout::parameterRenderer, and Help::calcLongOptionColumnWidth, making it easier to customize the table format used to lay out options and positional parameters in the usage help message.

CommandSpec API: added method CommandSpec::removeSubcommand.

This is the seventy-fifth public release.
Picocli follows semantic versioning.

Table of Contents

• New and noteworthy
  • New @PicocliScript2 annotation
  • Groovy Closures in Annotations
  • Key-only map parameters
  • System Properties
  • java.util.Optional<T>
  • Inherited Command Attributes
  • Preprocessor Parser Plugin
  • Model Transformations
• Fixed issues
• Deprecations
• Potential breaking changes

New and Noteworthy

New @PicocliScript2 annotation

The older @picocli.groovy.PicocliScript annotation is deprecated from picocli 4.6.
New scripts should use the @picocli.groovy.PicocliScript2 annotation (and associated picocli.groovy.PicocliBaseScript2 base class) instead.
The table below lists the differences between the PicocliBaseScript2 and PicocliBaseScript script base classes.

PicocliBaseScript2	PicocliBaseScript
Subcommands can be defined as @Command-annotated methods in the script.	No support for @Command-annotated methods.
Support for help subcommands (both the built-in one and custom ones).	No support for help subcommands.
Exit code support: scripts can override afterExecution(CommandLine, int, Exception) to call System.exit.	No support for exit code.
Invokes CommandLine::execute. Scripts can override beforeParseArgs(CommandLine) to install a custom IExecutionStrategy.	Execution after parsing is defined in PicocliBaseScript::run and is not easy to customize. Any subcommand and the main script are both executed.
Scripts can override beforeParseArgs(CommandLine) to install a custom IParameterExceptionHandler.	Invalid input handling can be customized by overriding PicocliBaseScript::handleParameterException.
Scripts can override beforeParseArgs(CommandLine) to install a custom IExecutionExceptionHandler.	Runtime exception handling can be customized by overriding PicocliBaseScript::handleExecutionException.
Implements Callable<Object>, script body is transformed to the call method.	Script body is transformed to the runScriptBody method.

Groovy Closures in Annotations

From picocli 4.6, Groovy programs can use closures in the picocli annotations instead of specifying a class.
This can be especially useful in Groovy scripts, where one cannot define a static inner class.

Example:

@Command(name = "ClosureDemo",
        versionProvider = {
            { -> ["line1" , "line2"] as String[] } as IVersionProvider // <1>
        },
        defaultValueProvider = {
            { argSpec -> "some default" } as IDefaultValueProvider // <2>
        })
class ClosureDemo {
    @Option(names = '-x', completionCandidates = {["A", "B", "C"]}) // <3>
    String x

    @Option(names = '-y',
            parameterConsumer = {
                { args, argSpec, commandSpec ->  // <4>
                    argSpec.setValue(args.toString() + commandSpec.name())
                    args.clear()
                } as IParameterConsumer
            })
    String y

    @Option(names = '-z', converter = [ // requires Groovy 3.0.7
            { { str -> MessageDigest.getInstance(str) } as ITypeConverter } // <5>
    ])
    MessageDigest z
}

When a class is specified, picocli creates an instance of the class. By contrast, when a closure is specified, picocli calls the closure to get an instance.
(To be precise, both of these are delegated to the configured factory, and the default factory implementation supports closures from picocli 4.6.)

As you can see in the above example, each closure in the annotation should contain another closure that has the required type (IVersionProvider, IDefaultValueProvider, etc.)

• <1> Command versionProvider: note the empty parameter list before the -> arrow. This is needed to help the Groovy compiler. The closure must be cast to IVersionProvider.
• <2> Command defaultProvider: return a default value for the specified ArgSpec parameter. The closure must be cast to IDefaultValueProvider.
• <3> Option or Parameters completionCandidates: return a list of Strings. No parameter list or casting is required.
• <4> Option or Parameters parameterConsumer: given a Stack, ArgSpec and CommandSpec, process the remaining arguments. The closure must be cast to IParameterConsumer.
• <5> Option or Parameters type converter takes an array of closures. Groovy 3.0.7 or greater is required: older versions of Groovy ignore closures in class array annotations. Each closure must have a parameter and be cast to ITypeConverter.

Key-only map parameters

By default, picocli expects Map options and positional parameters to look like key=value, that is, the option parameter or positional parameter is expected to have a key part and a value part, separated by a = character. If this is not the case, picocli shows a user-facing error message: Value for ... should be in KEY=VALUE format but was ....

From picocli 4.6, applications can specify a mapFallbackValue to allow end users to specify only the key part. The specified mapFallbackValue is put into the map when end users to specify only a key. The value type can be wrapped in a java.util.Optional. For example:

@Option(names = {"-P", "--properties"}, mapFallbackValue = Option.NULL_VALUE)
Map<String, Optional<Integer>> properties;

@Parameters(mapFallbackValue = "INFO", description = "... ${MAP-FALLBACK-VALUE} ...")
Map<Class<?>, LogLevel> logLevels;

This allows input like the following:

<cmd> --properties=key1 -Pkey2 -Pkey3=3 org.myorg.MyClass org.myorg.OtherClass=DEBUG

The above input would give the following results:

properties = [key1: Optional.empty, key2: Optional.empty, key3: Optional[3]]
logLevels  = [org.myorg.MyClass: INFO, org.myorg.OtherClass: DEBUG]

Note that the option description may contain the ${MAP-FALLBACK-VALUE} variable which will be replaced with the actual map fallback value when the usage help is shown.

System Properties

A common requirement for command line applications is to support the -Dkey=value syntax to allow end users to set system properties.

The example below uses the Map type to define an @Option-annotated method that delegates all key-value pairs to System::setProperty.
Note the use of mapFallbackValue = "" to allow key-only option parameters.

class SystemPropertiesDemo {
    @Option(names = "-D", mapFallbackValue = "") // allow -Dkey
    void setProperty(Map<String, String> props) {
        props.forEach((k, v) -> System.setProperty(k, v));
    }
}

java.util.Optional<T>

From version 4.6, picocli supports single-value types wrapped in a java.util.Optional container when running on Java 8 or higher.
If the option or positional parameter was not specified on the command line, picocli assigns the value Optional.empty() instead of null.
For example:

@Option(names = "-x")
Optional<Integer> x;

@Option(names = "-D", mapFallbackValue = Option.NULL_VALUE)
Map<String, Optional<Integer>> map;

WARNING: Picocli has only limited support for java.util.Optional types:
only single-value types, and the values in a Map (but not the keys!) can be wrapped in an Optional container.
java.util.Optional cannot be combined with arrays or other Collection classes.

Inherited Command Attributes

Picocli 4.6 adds support for inheriting @Command attributes with the scope = ScopeType.INHERIT annotation.
Commands with this scope have their @Command attributes copied to all subcommands (and sub-subcommands, to any level of depth).

When a subcommand specifies an explicit value in its @Command annotation, this value is used instead of the inherited value.
For example:

@Command(name = "app", scope = ScopeType.INHERIT,
        mixinStandardHelpOptions = true, version = "app version 1.0",
        header = "App header",
        description = "App description",
        footerHeading = "Copyright%n", footer = "(c) Copyright by the authors",
        showAtFileInUsageHelp = true)
class App implements Runnable {
    @Option(names = "-x") int x;

    public void run() { System.out.printf("Hello from app %d%n!", x); }

    @Command(header = "Subcommand header", description = "Subcommand description")
    void sub(@Option(names = "-y") int y) {
        System.out.printf("Hello app sub %d%n!", y);
    }
}

The app command in the above example has scope = ScopeType.INHERIT, so its @Command properties are inherited by the sub subcommand.

The sub subcommand defines its own header and description, so these are not inherited from the parent command.
The help message for the subcommand looks like this:

Subcommand header
Usage: app sub [-hV] [-y=<arg0>] [@<filename>...]
Subcommand description
      [@<filename>...]   One or more argument files containing options.
  -h, --help             Show this help message and exit.
  -V, --version          Print version information and exit.
  -y=<arg0>
Copyright
(c) Copyright by the authors

Note that the subcommand has inherited the mixed-in standard help options (--help and --version), the @file usage help, and the footer and footer heading.
It also inherited the version string, shown when the user invokes app sub --version.

When a command has scope = INHERIT, the following attributes are copied to its subcommands:

• all usage help attributes: description, descriptionHeading, header, headerHeading, footer, footerHeading, customSynopsis, synopsisHeading, synopsisSubcommandLabel, abbreviateSynopsis, optionListHeading, parameterListHeading, commandListHeading, exitCodeList, exitCodeListHeading, requiredOptionMarker, showDefaultValues, sortOptions, autoWidth, width, showAtFileInUsageHelp, showEndOfOptionsDelimiterInUsageHelp, and hidden
• exit codes: exitCodeOnSuccess, exitCodeOnUsageHelp, exitCodeOnVersionHelp, exitCodeOnInvalidInput, exitCodeOnExecutionException
• the help and version options mixed in by mixinStandardHelpOptions
• separator between option and option parameter
• version
• versionProvider
• defaultValueProvider
• subcommandsRepeatable
• whether this command is a helpCommand

Attributes that are not copied include:

• command name
• command aliases
• options and parameters (other than the help and version options mixed in by mixinStandardHelpOptions)
• other mixins than mixinStandardHelpOptions
• subcommands
• argument groups

Preprocessor Parser Plugin

Introduced in picocli 4.6, the IParameterPreprocessor is also a parser plugin, similar to IParameterConsumer, but more flexible.

Options, positional parameters and commands can be assigned a IParameterPreprocessor that implements custom logic to preprocess the parameters for this option, position or command.
When an option, positional parameter or command with a custom IParameterPreprocessor is matched on the command line, picocli's internal parser is temporarily suspended, and this custom logic is invoked.

This custom logic may completely replace picocli's internal parsing for this option, positional parameter or command, or augment it by doing some preprocessing before picocli's internal parsing is resumed for this option, positional parameter or command.

The "preprocessing" actions can include modifying the stack of command line parameters, or modifying the model.

Example use case

This may be useful when disambiguating input for commands that have both a positional parameter and an option with an optional parameter.
For example, suppose we have a command with the following synopsis:

edit [--open[=<editor>]] <file>

One of the limitations of options with an optional parameter is that they are difficult to combine with positional parameters.

With a custom parser plugin, we can customize the parser, such that VALUE in --option=VALUE is interpreted as the option parameter, and in --option VALUE (without the = separator), VALUE is interpreted as the positional parameter.
The code below demonstrates:

@Command(name = "edit")
class Edit {

    @Parameters(index = "0", description = "The file to edit.")
    File file;

    enum Editor { defaultEditor, eclipse, idea, netbeans }

    @Option(names = "--open", arity = "0..1", preprocessor = Edit.MyPreprocessor.class,
        description = {
            "Optionally specify the editor to use; if omitted the default editor is used. ",
            "Example: edit --open=idea FILE opens IntelliJ IDEA (notice the '=' separator)",
            "         edit --open FILE opens the specified file in the default editor"
        })
    Editor editor = Editor.defaultEditor;
    
    static class MyPreprocessor implements IParameterPreprocessor {
        public boolean preprocess(Stack<String> args,
                                  CommandSpec commandSpec,
                                  ArgSpec argSpec,
                                  Map<String, Object> info) {
            // we need to decide whether the next arg is the file to edit
            // or the name of the editor to use...
            if (" ".equals(info.get("separator"))) { // parameter was not attached to option

                // act as if the user specified --open=defaultEditor
                args.push(Editor.defaultEditor.name());
            }
            return false; // picocli's internal parsing is resumed for this option
        }
    }
}

With this preprocessor, the following user input gives the following command state:

# User input # Command State
# --------------------------
--open A B   # editor: defaultEditor, file: A,    unmatched: [B]
--open A     # editor: defaultEditor, file: A,    unmatched: []
--open=A B   # editor: A,             file: B,    unmatched: []
--open=A     # editor: A,             file: null, unmatched: []

Model Transformations

From picocli 4.6, it is possible to use the annotations API to modify the model (commands, options, subcommands, etc.) dynamically at runtime.
The @Command annotation now has a modelTransformer attribute where applications can specify a class that implements the IModelTransformer interface:

This allows applications to dynamically add or remove options, positional parameters or subcommands, or modify the command in any other way, based on some runtime condition.

@Command(modelTransformer = Dynamic.SubCmdFilter.class)
class Dynamic {

    private static class SubCmdFilter implements IModelTransformer {
        public CommandSpec transform(CommandSpec commandSpec) {
            if (Boolean.getBoolean("disable_sub")) {
                commandSpec.removeSubcommand("sub");
            }
            return commandSpec;
        }
    }

    @Command
    private void sub() {
        // subcommand business logic
    }
}

Fixed issues

• [#1164] API: Add support for @Command(scope=INHERIT). Thanks to Nick Cross for raising this.
• [#1191] API: Add @PicocliScript2 annotation to support subcommand methods in Groovy scripts. Thanks to Mattias Andersson for raising this.
• [#1241] API: Add mapFallbackValue attribute to @Options and @Parameters annotations, and corresponding ArgSpec.mapFallbackValue().
• [#1217] API: Add IParameterPreprocessor parser plugin to invoke custom logic when a command, option or positional parameter is matched. Thanks to David Walluck for raising this.
• [#1259][#1266] API: Add IModelTransformer to support user-defined model transformations after initialization and before parsing. Thanks to Marko Mackic for the pull request.
• [#802][#1284] API: Add support for echo and prompt in for interactive options and positional parameters. Thanks to H.Sakata for the pull request.
• [#1184] API: Added public methods Help.Layout::colorScheme, Help.Layout::textTable, Help.Layout::optionRenderer, Help.Layout::parameterRenderer, and Help::calcLongOptionColumnWidth.
• [#1254] API: Added ArgSpec::root: this method returns the original ArgSpec for inherited ArgSpec objects, and null for other ArgSpec objects. Thanks to Daniel Gray for the pull request.
• [#1256] API: Added CommandSpec::removeSubcommand method. Thanks to Marko Mackic for raising this.
• [#1258] API: Groovy programs can now use closures in the picocli annotations instead of specifying a class. Thanks to Adrian A. for raising this.
• [#1267] API: Add CommandLine::getFactory accessor for the factory. Thanks to Sualeh Fatehi for the suggestion.
• [#1108] Enhancement: Support Optional<T> type for options and positional parameters. Thanks to Max Rydahl Andersen for raising this.
• [#1214] Enhancement: Support Map options with key-only (support -Dkey as well as -Dkey=value). Thanks to Max Rydahl Andersen and David Walluck for raising this and subsequent discussion.
• [#1260] Enhancement: Support @Spec-annotated members in ArgGroup classes. Thanks to Jannick Hemelhof for raising this.
• [#1265] Enhancement in picocli-shell-jline3: add built-in clear command and improve help command. Thanks to Sualeh Fatehi for the pull request.
• [#1236] Enhancement/bugfix: Fix compiler warnings about Annotation::getClass and assignment in if condition. Thanks to nveeser-google for the pull request.
• [#1229] Bugfix: Fix compilation error introduced with fc5ef6d (#1184). Thanks to Andreas Deininger for the pull request.
• [#1225] Bugfix: Error message for unmatched positional argument reports an incorrect index when value equals a previously matched argument. Thanks to Vitaly Shukela for raising this.
• [#1250] Bugfix: Inherited positional parameter should not be overridden by default value if placed after subcommand. Thanks to Daniel Gray for the pull request.
• [#1183] Bugfix: Prevent MissingParameterException thrown when subcommand has required options and help option is specified on parent command. Thanks to drkilikil for raising this.
• [#1273] Bugfix: The Help.calcLongOptionColumnWidth now calls Help.createDefaultOptionRenderer, so overriding createDefaultOptionRenderer uses the correct column width in the options and parameters list.
• [#1274] Bugfix: ReflectionConfigGenerator should not generate method section in subclass config for private superclass methods in reflect-config.json. Thanks to Sebastian Thomschke for raising this.
• [#1215] DOC: User manual improvements, including more tabs with Kotlin source code. Thanks to Andreas Deininger for the pull request.
• [#1219] DOC: User manual improvements: added more tabs with Kotlin code. Thanks to Andreas Deininger for the pull request.
• [#1220] DOC: User manual improvements: corrections, more Kotlin tabs. Thanks to Andreas Deininger for the pull request.
• [#1221] DOC: User manual improvements: add tabs with Kotlin code for samples (chapter 14: Usage help). Thanks to Andreas Deininger for the pull request.
• [#1222] DOC: User manual improvements: add tabs with Kotlin code for samples (chapter 7 + 12). Thanks to Andreas Deininger for the pull request.
• [#1223] DOC: User manual improvements: add tabs with Kotlin code for samples (chapter 10: Validation). Thanks to Andreas Deininger for the pull request.
• [#1224] DOC: User manual improvements: add tabs with Kotlin code for samples (chapter 5: Default Values). Thanks to Andreas Deininger for the pull request.
• [#1226] DOC: User manual improvements: add tabs with Kotlin code for samples (chapter 9.6 - 9.8: Executing commands). Thanks to Andreas Deininger for the pull request.
• [#1228] DOC: User manual improvements: add tabs with Kotlin code for samples (chapters 8, 16, 20). Thanks to Andreas Deininger for the pull request.
• [#1230] DOC: User manual improvements: add tabs with Kotlin code for samples (Chapters 6, 11, 15, 19). Thanks to Andreas Deininger for the pull request.
• [#1232] DOC: User manual improvements for Micronaut example: add Kotlin version, extended description of Micronaut usage. Thanks to Andreas Deininger for the pull request.
• [#1233] DOC: User manual improvements: add tabs with Kotlin code for samples (Chapter 21: Tips & Tricks). Thanks to Andreas Deininger for the pull request.
• [#1234] DOC: add system properties example to user manual.
• [#1235][#1238] DOC: User manual: update DI section on Quarkus. Thanks to Andreas Deininger for the pull request.
• [#1246] DOC: User manual improvements: Guice and Spring Boot examples: add Kotlin versions. Thanks to Andreas Deininger for the pull request.
• [#1242] DOC: "Foldable" table of contents for User Manual and Quick Guide. Thanks to Andreas Deininger for the pull request.
• [#1247] DOC: User manual: extended Spring Boot example. Thanks to Andreas Deininger for the pull request.
• [#1249] DOC: Added section on Mocking to user manual. Thanks to Petr Hála for the pull request.
• [#1244] DEP: Bump Spring-Boot-Starter version to 2.3.5.RELEASE. Thanks to Andreas Deininger for the pull request.
• [#1289] DEP: Bump Spring boot dependency to 2.4.1. Thanks to Andreas Deininger for the pull request.
• [#1248] BUILD: Fix gradle warnings. Thanks to Andreas Deininger for the pull request.
• [#1280] BUILD: Remove trailing comment from gradle.properties to prevent build error. Thanks to David Walluck for raising this.
• [#1253] DOC: Fix line endings to LF in documentation files. Thanks to Daniel Gray for the pull request.
• [#1255] DOC: User manual and Quick Guide: add Groovy, Kotlin and Scala examples. Thanks to Andreas Deininger for the pull request.
• [#1261] DOC: User manual improvements: add Scala code samples. Thanks to Andreas Deininger for the pull request.
• [#1262] DOC: User manual: include jbang in the Build Tools section. Thanks to Max Rydahl Andersen for the pull request.
• [#1263] DOC: User manual: show build scripts in tabs. Thanks to Andreas Deininger for the pull request.
• [#1264] DOC: Fix broken links to GraalVm native image build configuration. Thanks to Andreas Deininger for the pull request.
• [#1005] DOC: Add link to the CookieTemple cli-java template README. Thanks to Lukas Heumos for getting this added to CookieTemple.
• [#1276] DOC: User manual: add section for "really executable JARs". Thanks to David Phillips for the pull request.
• [#1286] DOC: Fix: @ParentObject should be @ParentCommand. Thanks to Laurent Almeras for the pull request.
• [#1290] DOC: JLine: change keystroke syntax 'Ctl-D' to more common used syntax 'Ctrl-D'. Thanks to Andreas Deininger for the pull request.
• [#1270] TEST: Fix issue #1103 in Issue1225UnmatchedArgBadIndex. Thanks to David Walluck for the pull request.

Deprecations

From this release, the @picocli.groovy.PicocliScript annotation in the picocli-groovy module is deprecated in favor of @picocli.groovy.PicocliScript2, and the picocli.groovy.PicocliBaseScript class is deprecated in favor of picocli.groovy.PicocliBaseScript2.

Potential breaking changes

Added method isOptional() to the picocli.CommandLine.Model.ITypeInfo interface.