App with subcommand: reading arguments from @file fails with quoted option value

[deining]

Minimal working example:

AtTest.java:

import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Option;

@Command(name="mycmd", subcommands = { Subcommand.class })
public class AtTest implements Runnable {

    public static void main(String[] args) {
        CommandLine cmd = new CommandLine(new AtTest());
        cmd.execute(args);
    }

    @Override
    public void run() {
    }
}

@Command(name = "sub")
class Subcommand implements Runnable {

    @Option(names = { "-p" })
    String prefix = "";

    @Override
    public void run() {
        System.out.println(String.format("Prefix: %s", prefix));
    }
}

I can run the sub subcommand without problems:

$ java -cp "my.jar" AtTest sub -p="PREFIX"
Prefix: PREFIX

Now I want to run the subcommand via an @file. Thus I'm creating the needed @file whichs helds the previously used command line arguments:

Test.at:

sub -p="PREFIX"

When trying to execute the sub subcommand, I'm running into trouble:

$ java -cp "my.jar" AtTest @Test.at
Unmatched argument at index 2: 'PREFIX'
Usage: mycmd sub [-p=<prefix>]
  -p=<prefix>

Once I remove the quotes from the option value (sub -p=PREFIX), the program runs fine using the @file.

Is this the intended behaviour? Can I make use of quoted values inside an @file when using subcommands?

[remkop]

Nice catch!

Tracing shows this:

[picocli INFO] Expanded file @C:\Users\remko\IdeaProjects\picocli3\src\test\resources\argfile-issue1457.txt to arguments [sub, -p=, PREFIX]

So the picocli parser receives three arguments: sub, -p= and PREFIX.
This is then interpreted as the -p option having an empty string as the value, and PREFIX becomes an unmatched argument.

Internally picocli uses a StreamTokenizer for parsing at-files.
Looks like the tokenizer decides that a quoted value (even if attached to the previous token) constitutes a new token.

I don't like the prospect of creating a custom tokenizer, so perhaps the best way to deal with this is to document that any quoted value is interpreted as a separate argument, so instead of --option="VALUE", at-files should either use

--option "VALUE"

or

"--option=VALUE"

Thoughts?

[deining]

Thanks for pointing out what's internally going on here.

I don't like the prospect of creating a custom tokenizer

I fully agree, I don't like that apporach either.

so perhaps the best way to deal with this is to document that any quoted value is interpreted as a separate argument

I just submitted PR #1462 which extends the documentation of argument files.
This PR addresses another issue I came across when using UTF-8 encoded argument files:
You can easily break your app when changing the code page of your command line to UTF8:

c:\my\path>chcp 65001
Active code page: 65001
c:\my\path>java MyCommand @/home/foo/args
Exception in thread "main" java.nio.charset.UnsupportedCharsetException: cp65001
        at java.base/java.nio.charset.Charset.forName(Charset.java:529)
        at picocli.CommandLine.getStderrEncoding(CommandLine.java:14531)
        at picocli.CommandLine.getErr(CommandLine.java:1236)
        at picocli.CommandLine.handleUnhandled(CommandLine.java:2097)
        at picocli.CommandLine.execute(CommandLine.java:2093)
        at MyCommand.main(Unknown Source)

Fortunately, you can work around this by using a VM argument or by setting envoriment variable JAVA_TOOL_OPTIONS. The submitted PR documents this, too.

[remkop]

Wonderful, thank you for the PR!
I merged it, so I will close this ticket.
Many thanks again!