Make simplified @files JCommander-compatible

[triceo]

When we did #551 , we forgot about one corner case, and that is empty lines at the end. JCommander accepts files without them, our implementation does not.

[remkop]

What is the expected behaviour? Can you provide a failing unit test?

[triceo]

Will do.

[remkop]

I'm planning to do a picocli 3.9 release end of this week, maybe around December 29. Do you think you can provide a failing unit test before then?

[triceo]

@remkop OK, will take a look later today. Thanks for the extra motivation. :-)

[remkop]

Thanks for the PR! I'll take a look ASAP.

[remkop]

To be honest I had some trouble understanding what the problem was here...

If I apply the PR in #577 and run the tests, I see the testAtFileSimplified test fail like this:

picocli.CommandLine$UnmatchedArgumentException: Unmatched argument: 

	at picocli.CommandLine$Interpreter.parse(CommandLine.java:7166)
	at picocli.CommandLine$Interpreter.parse(CommandLine.java:7022)
	at picocli.CommandLine.parse(CommandLine.java:824)
	at picocli.CommandLine.populateCommand(CommandLine.java:777)
	at picocli.CommandLineTest.testAtFileSimplified(CommandLineTest.java:2850)

The reason it fails is that the parser finds an empty String as the last argument. The AtFileTestingApp command used in the test only defines named options, and does not define positional parameters. So, if the parser encounters a positional parameter (the empty string) it throws an UnmatchedArgumentException.

This seemed perfectly fine to me, and I did not understand what the problem was until I looked at the JCommander source. It appears that JCommander drops empty lines in argument files (@-files). This means it is not possible to specify empty string parameters in this format.

So if the goal is to make "simplified at files" behaviour compatible with JCommander, we should drop all empty lines, not just the trailing empty lines. Is this what you have in mind?

[triceo]

@remkop If you're saying that the behavior applies to all empty lines, then I agree that it makes sense to remove them as well.

(For context: I did not experience this issue myself. It was reported by one of my users, who used an old JCommander at-file with a newer version of my application, now using Picocli. And the problem the user demonstrated was the trailing empty line.)

[remkop]

Ok, I’ll change the behaviour of simplified at files to:

• ignore all empty lines
• trim the line before checking if it’s a comment line (starting with `#)

That will make the behaviour compatible with JCommander.

I’m also thinking to introduce API for this:

• CommandLine.isUseSimplifiedAtFiles
• CommandLine.setUseSimplifiedAtFiles
• ParserSpec.useSimplifiedAtFiles getter
• ParserSpec.useSimplifiedAtFiles setter

It may make sense to keep the system property but in the documentation mention the API instead and remove mentions of the system property.

Thoughts?

[triceo]

@remkop Sounds good. I appreciate the new API - I never really liked setting things through system properties.

[remkop]

The new parsing behaviour that ignores empty lines and comments with leading whitespace has been pushed to master. Thanks again for the PR with the test.

[remkop]

The additional API to control simplified @file format has been pushed to master.
Please verify if you have a chance.

[triceo]

@remkop I built the latest master a couple minutes ago and verified that the feature works for my use case.

[remkop]

Great, thanks for the confirmation!

[triceo]

@remkop Thanks for fixing the issue. Looking forward to 3.9.0!