built-in generate-completion subcommand: can't get it working

[deining]

I can't get the built-in generate-completion subcommand to work:

Example: 2 source files, one for main command and one for subcommand

Main command

package test;
import picocli.CommandLine;

public class AutoCompleteDemo {
    public static void main(String[] args) {
        CommandLine cmd = new CommandLine(new TestCommand());
        cmd.execute("generate-completion");
    }
}

Subcommand

package test;

import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Option;

@Command(name = "testcommand", subcommands = { CommandLine.HelpCommand.class, picocli.AutoComplete.GenerateCompletion.class },
mixinStandardHelpOptions = true, description = "test")
public class TestCommand implements Runnable {

    @Option(names="-n")
    int n;
    
    @Override
    public void run() { }
}

I can produce the completion script by invoking picocli.AutoComplete from the command line:

$ java -cp "AutoCompleteDemo.jar;picocli-4.1.0.jar"  picocli.AutoComplete -f  test.TestCommand
$ echo $?
0
$ ls testcommand_completion
testcommand_completion* 

However, I cannot produce the completion script by simply using the subcommand generate-completion:

$ java -cp "AutoCompleteDemo.jar;picocli-4.1.0.jar"  test.AutoCompleteDemo generate-completion
$ echo $?
0

The exit code is 0, however, after running the subcommand, I could not find any file inside the directory (debugging output is given below).

What am I doing wrong?

By the way: is there any way (--verbose switch or so) to get an idea what is running behind the scenes when generating the completion script? Would be really helpful!

Debugging output

$ java -cp "S.jar;picocli-4.1.0.jar"  test.AutoCompleteDemo
[picocli DEBUG] Creating CommandSpec for object of class test.TestCommand with factory picocli.CommandLine$DefaultFactory
[picocli DEBUG] Creating CommandSpec for object of class picocli.CommandLine$HelpCommand with factory picocli.CommandLine$DefaultFactory
[picocli DEBUG] Adding subcommand 'help' to 'testcommand'
[picocli DEBUG] Created Messages from resourceBundle[base=null] for command 'help' (picocli.CommandLine$HelpCommand)
[picocli DEBUG] Creating CommandSpec for object of class picocli.AutoComplete$GenerateCompletion with factory picocli.CommandLine$DefaultFactory
[picocli DEBUG] Creating CommandSpec for object of class picocli.CommandLine$AutoHelpMixin with factory picocli.CommandLine$DefaultFactory
[picocli DEBUG] Adding subcommand 'generate-completion' to 'testcommand'
[picocli DEBUG] Created Messages from resourceBundle[base=null] for command 'generate-completion' (picocli.AutoComplete$GenerateCompletion)
[picocli DEBUG] Creating CommandSpec for object of class picocli.CommandLine$AutoHelpMixin with factory picocli.CommandLine$DefaultFactory
[picocli INFO] Picocli version: 4.1.0, JVM: 11.0.1 (Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 11.0.1+13-LTS), OS: Windows 10 10.0 amd64
[picocli INFO] Parsing 1 command line args [generate-completion]
[picocli DEBUG] Parser configuration: posixClusteredShortOptionsAllowed=true, stopAtPositional=false, stopAtUnmatched=false, separator=null, overwrittenOptionsAllowed=false, unmatchedArgumentsAllowed=false, expandAtFiles=true, atFileCommentChar=#, useSimplifiedAtFiles=false, endOfOptionsDelimiter=--, limitSplit=false, aritySatisfiedByAttachedOptionParam=false, toggleBooleanFlags=false, unmatchedOptionsArePositionalParams=false, collectErrors=false,caseInsensitiveEnumValuesAllowed=false, trimQuotes=false, splitQuotedStrings=false
[picocli DEBUG] (ANSI is enabled by default: systemproperty[picocli.ansi]=null, isatty=false, TERM=xterm, OSTYPE=null, isWindows=true, JansiConsoleInstalled=false, ANSICON=null, ConEmuANSI=null, NO_COLOR=null, CLICOLOR=null, CLICOLOR_FORCE=null)
[picocli DEBUG] Set initial value for field int test.TestCommand.n of type int to 0.
[picocli DEBUG] Set initial value for field boolean picocli.CommandLine$AutoHelpMixin.helpRequested of type boolean to false.
[picocli DEBUG] Set initial value for field boolean picocli.CommandLine$AutoHelpMixin.versionRequested of type boolean to false.
[picocli DEBUG] Initializing test.TestCommand: 3 options, 0 positional parameters, 0 required, 0 groups, 2 subcommands.
[picocli DEBUG] Processing argument 'generate-completion'. Remainder=[]
[picocli DEBUG] Found subcommand 'generate-completion' (picocli.AutoComplete$GenerateCompletion)
[picocli DEBUG] Set initial value for field boolean picocli.CommandLine$AutoHelpMixin.helpRequested of type boolean to false.
[picocli DEBUG] Set initial value for field boolean picocli.CommandLine$AutoHelpMixin.versionRequested of type boolean to false.
[picocli DEBUG] Initializing picocli.AutoComplete$GenerateCompletion: 2 options, 0 positional parameters, 0 required, 0 groups, 0 subcommands.

[remkop]

This is a bug. Thank you for finding this! My tests did not show it because I was setting the destination stream to a StringWriter.

If the stream is not set, then a call to CommandLine.getOut() will create a new PrintWriter (wrapped around System.out) for every call to CommandLine.getOut. This internally has a buffer. The autocompletion command calls getOut three times: once for the script, once for the closing \n, and once to call flush(). This last call has no effect because it is a different PrintWriter instance - the script is still stuck in the buffer of the first PrintWriter...

It used to be simply getOut().println(script), but I replaced that with the current code to ensure that the command never prints a Windows line separator \r\n to the script (which can cause weird errors). However, I failed to test this with the standard out stream...

This exposes an existing problem with getOut. I will fix this ASAP.

The fix is that the first call to getOut should set the out field to a PrintWriter, and this PrintWriter should be used subsequently.

[remkop]

I released picocli 4.1.1 with the fix for this issue.
Thanks again for finding this!