allow_external_subcommands interacts weirdly with boolean flags?

[senekor]

Hi, I stumbled on a behavior that I did not understand. Here is a minimal reproduction:

use clap::{CommandFactory, FromArgMatches};

#[derive(clap::Parser, Clone, Debug)]
struct Args {
    #[arg(long)]
    bool_flag: bool,
}

fn main() {
    let app = Args::command().ignore_errors(true);
    let args = vec!["my-app", "--bool-flag", "whatever"];

    let matches = app
        .clone()
        .allow_external_subcommands(true)
        .get_matches_from(&args);
    println!(
        "set   with    allow_external_subcommands: {:?}",
        matches.get_one::<bool>("bool_flag")
    );

    let matches = app.clone().get_matches_from(&args);
    println!(
        "set   without allow_external_subcommands: {:?}",
        matches.get_one::<bool>("bool_flag")
    );

    let args = vec!["my-app", "whatever"];

    let matches = app
        .clone()
        .allow_external_subcommands(true)
        .get_matches_from(&args);
    println!(
        "unset with    allow_external_subcommands: {:?}",
        matches.get_one::<bool>("bool_flag")
    );

    let matches = app.clone().get_matches_from(&args);
    println!(
        "unset without allow_external_subcommands: {:?}",
        matches.get_one::<bool>("bool_flag")
    );

    Args::from_arg_matches(&matches).unwrap();
}

and the output of that is:

set   with    allow_external_subcommands: Some(true)
set   without allow_external_subcommands: Some(true)
unset with    allow_external_subcommands: Some(false)
unset without allow_external_subcommands: None
thread 'main' panicked at src/main.rs:47:38:
... ErrorInner { kind: MissingRequiredArgument ... message: Some(Raw("The following required argument was not provided: bool_flag")) ...
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

My question is, why does allow_external_subcommands cause an unset boolean flag to be parsed as Some(false), while it is parsed as None without it?

For completeness, here's the context of what I'm trying to do, though it's not that relevant to the question:

context

I'm working on dynamic completions for jj. Because of reasons, we shell out to ourselves in the custom completer functions. There are a couple global flags, like --repository, which are important to pass along to the subprocess. So we try to just parse those global args to pass them along.

My initial code was crashing like the above example. I just started experimenting with how I could get clap to parse my boolean flags as default-false if unset, which is what I would expect in any case. I landed on allow_external_subcommands, which seems to solve my problem, but I have no idea why it does that.

Please do let me know if I'm "holding it wrong" 🙂

[epage]

To more clearly see whats happening, I've switched it to using the builder API

#!/usr/bin/env nargo
---
[dependencies]
clap = { version = "4", features = ["derive"] }
---

use clap::{CommandFactory, FromArgMatches, Command, Arg, builder::ArgAction};

#[derive(clap::Parser, Clone, Debug)]
struct Args {
    #[arg(long)]
    bool_flag: bool,
}

fn app() -> clap::Command {
    Command::new("foo")
        .arg(Arg::new("bool_flag").long("bool-flag").action(ArgAction::SetTrue))
        .ignore_errors(true)
}

fn main() {
    let args = vec!["my-app", "--bool-flag", "whatever"];

    let matches = app()
        .allow_external_subcommands(true)
        .get_matches_from(&args);
    println!(
        "set   with    allow_external_subcommands: {:?}",
        matches.get_one::<bool>("bool_flag")
    );

    let matches = app().get_matches_from(&args);
    println!(
        "set   without allow_external_subcommands: {:?}",
        matches.get_one::<bool>("bool_flag")
    );

    let args = vec!["my-app", "whatever"];

    let matches = app()
        .allow_external_subcommands(true)
        .get_matches_from(&args);
    println!(
        "unset with    allow_external_subcommands: {:?}",
        matches.get_one::<bool>("bool_flag")
    );

    let matches = app().get_matches_from(&args);
    println!(
        "unset without allow_external_subcommands: {:?}",
        matches.get_one::<bool>("bool_flag")
    );

    Args::from_arg_matches(&matches).unwrap();
}

[epage]

The issue is more with ignore_errors which has a lot of unspecified behavior

#!/usr/bin/env nargo
---
[dependencies]
clap = { version = "4", features = ["derive"] }
---

use clap::{Command, Arg, builder::ArgAction};

fn app() -> clap::Command {
    Command::new("foo")
        .arg(Arg::new("bool_flag").long("bool-flag").action(ArgAction::SetTrue))
        //.ignore_errors(true)
}

fn parse(app: clap::Command, args: &[&str], case: &str) {
    match app.try_get_matches_from(args) {
        Ok(m) => {
            println!("{case}: {:?}", m.get_one::<bool>("bool_flag"));
        },
        Err(err) => {
            println!("{case}");
            let _ = err.print();
        },
    }
}

fn main() {
    let args = vec!["my-app", "--bool-flag", "whatever"];
    parse(
        app().allow_external_subcommands(true),
        &args,
        "set   with    allow_external_subcommands",
    );
    parse(
        app(),
        &args,
        "set   without allow_external_subcommands",
    );

    let args = vec!["my-app", "whatever"];
    parse(
        app().allow_external_subcommands(true),
        &args,
        "unset with    allow_external_subcommands",
    );
    parse(
        app(),
        &args,
        "unset without allow_external_subcommands",
    );
}

With ignore_errors(true)

$ ./clap-5812.rs
warning: `package.edition` is unspecified, defaulting to `2021`
   Compiling clap-5812 v0.0.0 (/home/epage/src/personal/dump)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.24s
     Running `/home/epage/.cargo/target/87/6e4db5a1c07ebe/debug/clap-5812`
set   with    allow_external_subcommands: Some(true)
set   without allow_external_subcommands: Some(true)
unset with    allow_external_subcommands: Some(false)
unset without allow_external_subcommands: None

Without ignore_errors(true)

$ ./clap-5812.rs
warning: `package.edition` is unspecified, defaulting to `2021`
   Compiling clap-5812 v0.0.0 (/home/epage/src/personal/dump)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.24s
     Running `/home/epage/.cargo/target/87/6e4db5a1c07ebe/debug/clap-5812`
set   with    allow_external_subcommands: Some(true)
set   without allow_external_subcommands
error: unexpected argument 'whatever' found

Usage: my-app [OPTIONS]

For more information, try '--help'.
unset with    allow_external_subcommands: Some(false)
unset without allow_external_subcommands
error: unexpected argument 'whatever' found

Usage: my-app [OPTIONS]

For more information, try '--help'.

With my-app --bool-flag whatever, --bool-flag is processed before any error associated with whatever.

With my-app whatever, default values are not being set when at least some errors occur.

[senekor]

That makes sense, thank you!

[epage]

Posted #5813 to fix this