Missing default value in ArgGroup. #1409

sbernard31 · 2021-08-13T15:23:54Z

I understand that by default defaultValue in ArgGroup are not applied.

The behavior is explained at : #876 (comment)
(Note: The default behavior seems surprising to me at first sight but maybe it's justified)

I faced a situation where it seems that this behavior is not respected (but maybe I missed something)
Sorry for the not so short example, I was not able to make it smaller.

The goal is to have this kind of options : X AND ((1A AND 1B) OU (2A OU 2B))
I set default value for X, 2A and 2B.
I share the code :

@Command(name = "myCommand",
         mixinStandardHelpOptions = true,
         description = "A command with default value in section ?")
public class Example implements Runnable {

    @ArgGroup(exclusive = false,
              heading = "%n@|italic " //
                      + "Options to be used with group 1 OR group 2 options." //
                      + "|@%n")
    public OptXAndGroupOneOrGroupTwo optXAndGroupOneOrGroupTwo = new OptXAndGroupOneOrGroupTwo();

    public static class OptXAndGroupOneOrGroupTwo {
        @Option(names = { "-x", "--option-x" }, required = true, defaultValue = "Default X", description = "option X")
        private String x;

        @ArgGroup(exclusive = true)
        private OneOrTwo oneORtwo = new OneOrTwo();
    }

    public static class OneOrTwo {

        @ArgGroup(exclusive = false,
                  heading = "%n@|bold Group 1|@ %n%n"//
                          + "@|italic " //
                          + "Description of the group 1 ." //
                          + "|@%n")
        public GroupOne one = new GroupOne();

        @ArgGroup(exclusive = false,
                  heading = "%n@|bold Group 2|@ %n%n"//
                          + "@|italic " //
                          + "Description of the group 2 ." //
                          + "|@%n")
        public GroupTwo two = new GroupTwo();
    }

    public static class GroupOne {
        @Option(names = { "-1a", "--option-1a" },required=true,description = "option A of group 1")
        private String _1a;

        @Option(names = { "-1b", "--option-1b" },required=true,description = "option B of group 1")
        private String _1b;
    }

    public static class GroupTwo {

        @Option(names = { "-2a", "--option-2a" },required=true, defaultValue = "Default 2A", description = "option A of group 2")
        private String _2a;

        
        @Option(names = { "-2b", "--option-2b" },required=true, defaultValue = "Default 2B", description = "option B of group 2")
        private String _2b;
    }

    public void run() {
        System.out.println();
        System.out.println(" X = " + optXAndGroupOneOrGroupTwo.x);
        System.out.println("1A = " + optXAndGroupOneOrGroupTwo.oneORtwo.one._1a);
        System.out.println("1B = " + optXAndGroupOneOrGroupTwo.oneORtwo.one._1b);
        System.out.println("2A = " + optXAndGroupOneOrGroupTwo.oneORtwo.two._2a);
        System.out.println("2B = " + optXAndGroupOneOrGroupTwo.oneORtwo.two._2b);
    }

    public static void main(String... args) {
        Example example = new Example();
        int exitCode = new CommandLine(example).execute(args);

        System.exit(exitCode);
    }
}

When I run my command, it outputs value of each options.
E.g if I run it without any options :

 X = Default X
1A = null
1B = null
2A = Default 2A
2B = Default 2B

Now if I run it just with option -x : myCommand -x ANOTHER_VALUE, I get :

 X = ANOTHER_VALUE
1A = null
1B = null
2A = null
2B = null

Default value of 2A and 2B are lost I don't know if this is expected but this is a surprising behavior.

The text was updated successfully, but these errors were encountered:

sbernard31 · 2021-08-13T15:27:05Z

Not really linked to this issue but as 1A, 1B, 2A and 2B are required, I expected that when X is used (1A AND 1B) or (2A or 2B) will be required.

remkop · 2021-08-14T07:40:08Z

Can you run -x ANOTHERVALUE again with -Dpicocli.trace=DEBUG?
I would like to see the output.

sbernard31 · 2021-08-16T09:15:41Z

[picocli DEBUG] Creating CommandSpec for test.picocli.examples.defaultvalue.Example@77c1870e with factory picocli.CommandLine$DefaultFactory
[picocli DEBUG] Creating CommandSpec for picocli.CommandLine$AutoHelpMixin@7371b5d5 with factory picocli.CommandLine$DefaultFactory
[picocli INFO] Picocli version: 4.6.1, JVM: 1.8.0_212 (Oracle Corporation OpenJDK 64-Bit Server VM 25.212-b03), OS: Linux 5.10.0-0.bpo.3-amd64 amd64
[picocli INFO] Parsing 2 command line args [-x, ANOTHER_VALUE]
[picocli DEBUG] Parser configuration: optionsCaseInsensitive=false, subcommandsCaseInsensitive=false, abbreviatedOptionsAllowed=false, abbreviatedSubcommandsAllowed=false, aritySatisfiedByAttachedOptionParam=false, atFileCommentChar=#, caseInsensitiveEnumValuesAllowed=false, collectErrors=false, endOfOptionsDelimiter=--, expandAtFiles=true, limitSplit=false, overwrittenOptionsAllowed=false, posixClusteredShortOptionsAllowed=true, separator=null, splitQuotedStrings=false, stopAtPositional=false, stopAtUnmatched=false, toggleBooleanFlags=false, trimQuotes=false, unmatchedArgumentsAllowed=false, unmatchedOptionsAllowedAsOptionParameters=true, unmatchedOptionsArePositionalParams=false, useSimplifiedAtFiles=false
[picocli DEBUG] (ANSI is enabled by default: systemproperty[picocli.ansi]=null, isatty=true, TERM=xterm-256color, OSTYPE=null, isWindows=false, JansiConsoleInstalled=false, ANSICON=null, ConEmuANSI=null, NO_COLOR=null, CLICOLOR=null, CLICOLOR_FORCE=null)
[picocli DEBUG] Initializing command 'myCommand' (user object: test.picocli.examples.defaultvalue.Example@77c1870e): 7 options, 0 positional parameters, 0 required, 1 groups, 0 subcommands.
[picocli DEBUG] Set initial value for field boolean picocli.CommandLine$AutoHelpMixin.helpRequested of type boolean to false.
[picocli DEBUG] Set initial value for field boolean picocli.CommandLine$AutoHelpMixin.versionRequested of type boolean to false.
[picocli DEBUG] [0] Processing argument '-x'. Remainder=[ANOTHER_VALUE]
[picocli DEBUG] '-x' cannot be separated into <option>=<option-parameter>
[picocli DEBUG] Found option named '-x': field String test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo.x, arity=1
[picocli INFO] Adding match to GroupMatchContainer [[-x=<x>] [[-1a=<_1a> -1b=<_1b>] | [[-2a=<_2a>] [-2b=<_2b>]]]]={} (group=1 [[-x=<x>] [[-1a=<_1a> -1b=<_1b>] | [[-2a=<_2a>] [-2b=<_2b>]]]]).
[picocli DEBUG] Creating new user object of type class test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo for group [[-x=<x>] [[-1a=<_1a> -1b=<_1b>] | [[-2a=<_2a>] [-2b=<_2b>]]]]
[picocli DEBUG] Created test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo@4c9458f6, invoking setter FieldBinding(test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo test.picocli.examples.defaultvalue.Example.optXAndGroupOneOrGroupTwo) with scope test.picocli.examples.defaultvalue.Example@77c1870e
[picocli DEBUG] Initializing --option-x=<x> in group [[-x=<x>] [[-1a=<_1a> -1b=<_1b>] | [[-2a=<_2a>] [-2b=<_2b>]]]]: setting scope to user object test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo@4c9458f6 and initializing initial and default values
[picocli DEBUG] Set initial value for field String test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo.x of type class java.lang.String to null.
[picocli DEBUG] Applying defaultValue (Default X) to field String test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo.x on OptXAndGroupOneOrGroupTwo@4c9458f6
[picocli DEBUG] 'Default X' doesn't resemble an option: 0 matching prefix chars out of 14 option names
[picocli INFO] Setting field String test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo.x to 'Default X' (was 'null') for field String test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo.x on OptXAndGroupOneOrGroupTwo@4c9458f6
[picocli DEBUG] Setting scope for subgroup [[-1a=<_1a> -1b=<_1b>] | [[-2a=<_2a>] [-2b=<_2b>]]] with setter=FieldBinding(test.picocli.examples.defaultvalue.Example$OneOrTwo test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo.oneORtwo) in group [[-x=<x>] [[-1a=<_1a> -1b=<_1b>] | [[-2a=<_2a>] [-2b=<_2b>]]]] to user object test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo@4c9458f6
[picocli DEBUG] Initialization complete for group [[-x=<x>] [[-1a=<_1a> -1b=<_1b>] | [[-2a=<_2a>] [-2b=<_2b>]]]]
[picocli DEBUG] 'ANOTHER_VALUE' doesn't resemble an option: 0 matching prefix chars out of 14 option names
[picocli INFO] Setting field String test.picocli.examples.defaultvalue.Example$OptXAndGroupOneOrGroupTwo.x to 'ANOTHER_VALUE' (was 'Default X') for option -x on OptXAndGroupOneOrGroupTwo@4c9458f6
[picocli DEBUG] Applying default values for command 'myCommand'
[picocli DEBUG] defaultValue not defined for field boolean picocli.CommandLine$AutoHelpMixin.helpRequested
[picocli DEBUG] defaultValue not defined for field boolean picocli.CommandLine$AutoHelpMixin.versionRequested
[picocli DEBUG] Applying default values for group '[[-x=<x>] [[-1a=<_1a> -1b=<_1b>] | [[-2a=<_2a>] [-2b=<_2b>]]]]'
[picocli DEBUG] Applying default values for group '[[-1a=<_1a> -1b=<_1b>] | [[-2a=<_2a>] [-2b=<_2b>]]]'
[picocli DEBUG] Applying default values for group '[-1a=<_1a> -1b=<_1b>]'
[picocli DEBUG] defaultValue not defined for field String test.picocli.examples.defaultvalue.Example$GroupOne._1a
[picocli DEBUG] defaultValue not defined for field String test.picocli.examples.defaultvalue.Example$GroupOne._1b
[picocli DEBUG] Applying default values for group '[[-2a=<_2a>] [-2b=<_2b>]]'
[picocli DEBUG] Applying defaultValue (Default 2A) to field String test.picocli.examples.defaultvalue.Example$GroupTwo._2a on GroupTwo@77553f0b
[picocli DEBUG] 'Default 2A' doesn't resemble an option: 0 matching prefix chars out of 14 option names
[picocli INFO] Setting field String test.picocli.examples.defaultvalue.Example$GroupTwo._2a to 'Default 2A' (was 'null') for field String test.picocli.examples.defaultvalue.Example$GroupTwo._2a on GroupTwo@77553f0b
[picocli DEBUG] Applying defaultValue (Default 2B) to field String test.picocli.examples.defaultvalue.Example$GroupTwo._2b on GroupTwo@77553f0b
[picocli DEBUG] 'Default 2B' doesn't resemble an option: 0 matching prefix chars out of 14 option names
[picocli INFO] Setting field String test.picocli.examples.defaultvalue.Example$GroupTwo._2b to 'Default 2B' (was 'null') for field String test.picocli.examples.defaultvalue.Example$GroupTwo._2b on GroupTwo@77553f0b

 X = ANOTHER_VALUE
1A = null
1B = null
2A = null
2B = null

MadFoal · 2021-10-30T23:08:40Z

@remkop I have been working on this issue. If I resolve it I will open a PR.

remkop · 2021-10-31T06:21:17Z

@MadFoal that would be great, thank you!

MadFoal · 2021-11-04T03:14:10Z

Update 1- looks like the default values do get properly stored in argSpec inside the applyGroupDefaults function correctly. There is something about the way it gets merged back into this that is causing an issue.

MadFoal · 2021-11-07T23:13:57Z

Updated 2-

"Picocli will not initialize the @ArgGroup-annotated field (and so no default values are applied) if none of the group options is specified on the command line." (https://picocli.info/#_default_values_in_argument_groups)

If you do not specify at least one value within the argument group, it will not instantiate the default values, which is the observed behavior for this case. Technically the program is working as advertised.

remkop · 2021-11-08T05:32:32Z

@MadFoal based on the debug output, I think there is a problem:

Applying defaultValue (Default 2A) to field String test.picocli.examples.defaultvalue.Example$GroupTwo._2a on GroupTwo@77553f0b
...

After parsing is done, picocli is trying to apply default values to all unmatched options in the command, where possible. Since the -2a and -2b options are in a group (GroupTwo) whose user object already has an instance (see below), it should be possible to apply default values.

@ArgGroup(exclusive = false, ...) 
public GroupTwo two = new GroupTwo();

However, in the debugger I can see that the default values are applied to a different instance of the GroupTwo class.

What I believe is happening is the following: initially, the OptionSpec for the -2a and -2b options are created with a reflective reference to the annotated field, and a reference to the original GroupTwo instance. When the -x option is matched, a new instance of OptXAndGroupOneOrGroupTwo, OneOrTwo and GroupTwo is created, but the OptionSpec for the -2a and -2b options within those subgroups still have references to the old GroupTwo instance. The GroupTwo reference is updated when the -2a and -2b options are matched on the command line (that is why I did not notice this issue before). But for default values, the reference is not updated, so the default values are applied to the original (now outdated) GroupTwo instance.

I need to dig a bit deeper into what would be the best way to fix this, but I do think the current behaviour is incorrect.

remkop · 2021-11-08T09:05:40Z

This is proving trickier than I thought...
When the -x option is matched, a new OptXAndGroupOneOrGroupTwo instance is created. Subgroups are updated to point to the new OptXAndGroupOneOrGroupTwo instance, but sub-subgroups are not updated.

When an options in a subgroup is matched, a new user object (e.g. a GroupTwo instance) for that subgroup is created, and this new object replaces the old reference; hence things work fine for arguments matched on the command line.

For default values there is no match, so this above logic is not invoked.
I have not yet found a way to fix this.

One idea was (when the new OptXAndGroupOneOrGroupTwo instance is created) to descend into the subgroups and update their object pointers but this is not trivial; while the group/subgroup hierarchy has not changed, the object hierarchy would need to be rediscovered via reflection. This would not work for programmatic models that do not contain annotated fields/methods.
Another idea was to invoke the same logic for applying default values as for when an option was matched; this does not work because it appears as if the option was actually matched, giving validation errors (for exclusive options etc.)

So while I can see that from @sbernard31's point of view the current defaulting behaviour is unsatisfactory, I am not sure this can be fixed without major rework, which I am currently not eager to do...

MadFoal · 2021-11-11T02:24:12Z

@remkop Okay thank you for that explanation. I thought it was along those lines, however I am significantly less familiar with the code. I noticed early that we were adding the default values but I was struggling with why they are not making it into the correct place. This makes more sense.

I had been looking at how to change "applyDefaultValues" to look for the subgroup created by the -x option and add default values if the current value is null. I believe this is what you are describing as "Another idea was to invoke the same logic for applying..."

There is another way of fixing this default behavior also less elegant. In the command declaration we hardcode the default values as such:
private String _2b = "Default 2B";
instead of
private String _2b;

remkop · 2021-11-11T03:29:35Z

@MadFoal I am not sure there is a true fix.
One thing to note is that picocli can only show the default value of options in arg groups when the default value is specified in the annotation (it cannot derive the default value from the field's initial value like it can when showing help for options that are not in an arg group).

The documentation for default values in arg groups states the situation reasonably well.

Applications should probably:

put any default values in the option annotation so that picocli can display them in the usage help message
groups with subgroups should probably not instantiate the group class in the declaration (so, use @ArgGroup MyGroup g; instead of @ArgGroup MyGroup g = new MyGroup(); ). This allows applications to determine whether an option in the group was matched on the command line: the @ArgGroup-annotated field has a non-null value when a group option was matched.
Assign default values manually: if the group was not matched (the annotated field is null), then assign default values in the application logic. @MadFoal's suggestion to put default values in the declaration of the annotated option fields may be the most concise way to do this; the application then only needs to do something like if (g == null) { g = new MyGroup(); } to assign default values to all options in the group. For groups with subgroups, applications may need to do the same for the nested subgroups. For example, g = new MyGroup(); g.sub = new Subgroup();. And yes, that means some duplication, because default values would need to be specified both in the annotations as well as in the option field initial value.

MadFoal · 2021-11-11T03:33:20Z

@remkop
Yes Sir, that is correct.

This is one way of overcoming the issue.

Do you intend to update the documentation for this issue?

remkop · 2021-11-11T05:57:59Z

@MadFoal Will you be able to provide a pull request to improve the documentation?

MadFoal · 2021-11-11T12:19:46Z

@remkop I can do that.

MadFoal · 2021-11-12T05:15:13Z

Documentation is coming along. Should have something tangible to submit shortly.

remkop · 2021-11-12T06:10:37Z

Great, thank you for working on this!
Some nitpicking (I am very very picky when it comes to the documentation! 😅 ):

please spell the library as picocli (all lower case)
example code should be as minimalistic/short as possible, only leave what is absolutely necessary:
- remove all public and private modifiers (except for the main method - then again, do we need the main method?)
- ultrashort meaningful names (ArgGroupDefaultValueTest -> MyApp, _a -> a, OptionXAndGroup -> Outer, TwoArgGroup -> Inner)
- don't qualify class names: ArgGroupDefaultValue.TwoArgGroup twoArgGroup = new ArgGroupDefaultValue.TwoArgGroup(); -> Inner inner;
- do we need required = true in this example?
- can we reduce the @Option annotation to one line? We can remove the long option name and the description, and make the default value shorter, like "AAA".

My recommendation to app authors using groups would be to not instantiate groups in their declaration:

// don't do this
class Outer {
    // this way, applications cannot tell whether an option in the Inner group was specified on the command line
    @ArgGroup Inner inner = new Inner();
}

but instead to just declare the field, so that it remains null when none of the options in the group were matched on the command line:

// do this instead
class Outer {
    @ArgGroup Inner inner; // this field will only be non-null if an option in this group was matched
}

The options in the groups can be initialized in their declaration, but should also have a default value in their annotation:

class Inner {
    // yes, there is some duplication here, unfortunately
    @Option(names = "-a", defaultValue = "AAA") String a = "AAA";
    @Option(names = "-b", defaultValue = "BBB") String b = "BBB";
}

It may be a good idea to add a run method to the example that shows what the application logic needs to do.

// MyApp
public void run() {
    if (outer == null) { // no options in the group were specified on the command line; apply defaults
        outer = new Outer();
    }
    if (outer.inner == null) { // same for nested sub-groups
        outer.inner = new Inner(); // apply the default values of the inner group
    }
    // remaining business logic...
}

MadFoal · 2021-11-12T06:37:27Z

I'll make these changes and re-submit.

MadFoal · 2021-11-12T20:52:14Z

PR #1463 with the changes you recommended. Let me know if there are more changes that I need to make.

This was referenced Aug 13, 2021

[ideas] MultiOption Validation #1410

Closed

[Ideas] Custom ArgGroup Validation/Rules #1411

Closed

remkop added the type: bug 🐛 label Oct 31, 2021

remkop added a commit that referenced this issue Nov 8, 2021

[#1409] add failing test for investigation purposes

4769e43

remkop linked a pull request Nov 13, 2021 that will close this issue

1409 documentation tests #1463

Merged

remkop closed this as completed in #1463 Nov 16, 2021

remkop added a commit that referenced this issue Nov 16, 2021

[#1409][#1463] DOC: update RELEASE-NOTES.md for new user manual section

a7be845

remkop added a commit that referenced this issue Nov 16, 2021

[#1409][#1463] DOC: update

cf4535b

remkop added a commit that referenced this issue Nov 17, 2021

[#1409][#1463] DOC: minor tweaks

33537c7

remkop added a commit that referenced this issue Nov 17, 2021

[#1409][#1463] DOC: fix typo

70f91b7

remkop added a commit that referenced this issue Nov 17, 2021

[#1409][#1463] DOC: more minor improvements

de9df3e

remkop added a commit that referenced this issue Nov 17, 2021

[#1409][#1463] DOC: more minor improvements

2857f7c

remkop added a commit that referenced this issue Nov 17, 2021

[#1409][#1463] DOC: more minor improvements

d7295dd

remkop added a commit that referenced this issue Nov 17, 2021

[#1409][#1463] DOC: fix typo

f35ea1e

remkop added a commit that referenced this issue Nov 17, 2021

[#1409][#1463] DOC: update generated HTML

14ff78b

remkop mentioned this issue May 20, 2022

Bug in initialisation of options grouped in classes (ArgGroup) #1685

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Missing default value in ArgGroup. #1409

Missing default value in ArgGroup. #1409

sbernard31 commented Aug 13, 2021

sbernard31 commented Aug 13, 2021

remkop commented Aug 14, 2021

sbernard31 commented Aug 16, 2021

MadFoal commented Oct 30, 2021

remkop commented Oct 31, 2021

MadFoal commented Nov 4, 2021

MadFoal commented Nov 7, 2021 •

edited

Loading

remkop commented Nov 8, 2021 •

edited

Loading

remkop commented Nov 8, 2021 •

edited

Loading

MadFoal commented Nov 11, 2021 •

edited

Loading

remkop commented Nov 11, 2021

MadFoal commented Nov 11, 2021 •

edited

Loading

remkop commented Nov 11, 2021

MadFoal commented Nov 11, 2021

MadFoal commented Nov 12, 2021 •

edited

Loading

remkop commented Nov 12, 2021 •

edited

Loading

MadFoal commented Nov 12, 2021

MadFoal commented Nov 12, 2021 •

edited

Loading

Missing default value in ArgGroup. #1409

Missing default value in ArgGroup. #1409

Comments

sbernard31 commented Aug 13, 2021

sbernard31 commented Aug 13, 2021

remkop commented Aug 14, 2021

sbernard31 commented Aug 16, 2021

MadFoal commented Oct 30, 2021

remkop commented Oct 31, 2021

MadFoal commented Nov 4, 2021

MadFoal commented Nov 7, 2021 • edited Loading

remkop commented Nov 8, 2021 • edited Loading

remkop commented Nov 8, 2021 • edited Loading

MadFoal commented Nov 11, 2021 • edited Loading

remkop commented Nov 11, 2021

MadFoal commented Nov 11, 2021 • edited Loading

remkop commented Nov 11, 2021

MadFoal commented Nov 11, 2021

MadFoal commented Nov 12, 2021 • edited Loading

remkop commented Nov 12, 2021 • edited Loading

MadFoal commented Nov 12, 2021

MadFoal commented Nov 12, 2021 • edited Loading

MadFoal commented Nov 7, 2021 •

edited

Loading

remkop commented Nov 8, 2021 •

edited

Loading

remkop commented Nov 8, 2021 •

edited

Loading

MadFoal commented Nov 11, 2021 •

edited

Loading

MadFoal commented Nov 11, 2021 •

edited

Loading

MadFoal commented Nov 12, 2021 •

edited

Loading

remkop commented Nov 12, 2021 •

edited

Loading

MadFoal commented Nov 12, 2021 •

edited

Loading