Is strict actually a goal?

[shadowspawn]

strict {Boolean} (Optional) A Boolean on wheather or not to throw an error when unknown args are encountered

The current interface is leaning towards the minimal (magic) configuration goal of the original proposal. I do not currently see a way of declaring an option as being known but not taking a value, so there is no way to identify unknown options?

The FAQ includes:

Do unknown arguments raise an error? Are they parsed? Are they treated as positional arguments?
no, they are parsed, not treated as positionals

Related:

• Default Behavior: Boolean Flags in util: add util.parseArgs() nodejs/node#35015 (comment)
• util: add util.parseArgs() nodejs/node#35015 (comment)
• util: add util.parseArgs() nodejs/node#35015 (comment)
• util: add util.parseArgs() nodejs/node#35015 (comment)

(A different case is --foo=bar when foo is not specified to take an argument value. Not including that as part of strict for current question. nodejs/node#35015 (comment))

[ljharb]

It should be a goal to error and show help output when an unknown argument is passed, just like when a known argument is passed with an incorrect value.

[darcyclarke]

@ljharb

"It should be a goal to error and show help output when an unknown argument is passed"

Agree.

"just like when a known argument is passed with an incorrect value."

Disagree. Also, a bit confused; I don't see any way the current spec allows you to define a value which validates against a value passed. The goal of this spec, which differentiated it from the previous work, was that you'd "bring your own validation" in reference to values & types.

@shadowspawn the F.A.Q. is fairly conversational in nature (as it literally spawned out of a discussion between myself & @isaacs); with that in mind, it's correct in stating that we should parse (to the best of our abilities & with the options provided) all input. strict mode should throw an error when args are found that aren't explicitly defined/"known" in withValues (or... maybe some other option which isn't spec'd yet?)

[darcyclarke]

(A different case is --foo=bar when foo is not specified to take an argument value. Not including that as part of strict for current question. nodejs/node#35015 (comment))

This makes sense... I'm actually a +1 to just kill strict for now. End-user can decide to throw if they see a flag they weren't expecting.

[ljharb]

@darcyclarke i meant conceptually - obviously this package wouldn't provide validation, but what's the use case for anyone wanting to permit unknown values?

[darcyclarke]

@ljharb I know of usecases, today, where you'd want to capture unknown/undefined flags (to the root program) & pass those along to another process or use in some form later (ex. npm's support of infinite config 👀 🤦🏻). Not saying it's a best practice, just that usecases exist.

Contrived Example:

node mkdir.js ./path/to/new/dir/ --force --verbose --parents

// mkdir.js

const knownOpts = ['force']
const { flags, positionals } = parseArgs({ withValue: knownOpts })
const args = Object.keys(flags).filter(f => knownOpts[f])
const cmd = (flags.force) ? 'sudo mkdir' : 'mkdir'

process('child_process').spawnSync(cmd, [...args, ...positionals])

In the above, I expect & am aware of --force but all other flags & positionals get passed along to a child process (note: mkdir will then throw if it's not aware of those).

[ljharb]

Must we support every possible use case? Or perhaps it’d be better to establish good defaults, since nobody is forced to use this solution, and it’s perfectly fine if not everyone can.

[bcoe]

Are we okay with dropping strict behavior for MVP? we can change this decision depending on code review on Node.js.

[ljharb]

I'm still not sure why anyone would want non-strict - it seems much better to be strict by default, and let developers opt in to allowing unknown arguments.

[shadowspawn]

Reference, two of the summary message from previous thread covering strictness and correctness:

• util: add util.parseArgs() nodejs/node#35015 (comment)
• util: add util.parseArgs() nodejs/node#35015 (comment)

[shadowspawn]

Short version: I am ok with wrapping a release without strict detection of unknown arguments. But I am willing to invest in strict mode for the misuse errors in first instance, and then for unknowns.

Long version

A big part of the original vision is the minimal config. This is incompatible with strict since flags are discovered at runtime and can't be "unknown", typos can not be detected by the parser.

I think the minimal config is appealing and useful for quick programs used mainly by their own author. I am reassured that @bcoe thinks the non-strict mode is useful with his experience with Yargs and support issues.

To make parseArgs useful to a wider audience, I think strict is important. This expands the audience to safer usage where the author can rely on common errors being detected and handled without writing their own code, and the end-user can rely on the program blocking many cases of accidental misuse. I want the utilities I write to become strict as they mature.

I don't feel it is worth silently returning an error message when not strict. (This idea is raised in a few places in the context of strict, and idea that parsing always succeeds.) I think that once an error has been detected then the parsing is tainted. So strict is a yes/no mode, the strict parse succeeded or it failed, and an exception is appropriate.

I suggest as a proof of concept we implement strict to throw for the error cases for a flag used with a value, and withValue option used without a value. These are within the scope of what is already described in the README and encountered in the current code and discussed at length in open issues. This is not what the README describes strict as for, but I feel it is within the spirit!

(As an aside, I continue to be impressed by how flexible Minimist is. Looks like is is capable of implementing strict for unknown arguments by using opts.unknown and declaring the expected options.)

[ljharb]

I’ve used yargs in a half dozen projects, and always ended up adding strict later. Without strict, adding any argument is a breaking change. Strict mode is the only way semver-minor becomes practical.

[bcoe]

I’ve used yargs in a half dozen projects, and always ended up adding strict later.
I suggest as a proof of concept we implement strict to throw for the error cases for a flag used with a value, and withValue

I'm convinced we should have strict for MVP, but I'd make the case for it being an opt in, as with yargs.

Would you be okay with this compromise @ljharb (needing to set a strict: true in the options).

[ljharb]

@bcoe can you actually make the case? in what use case or realm of computing is non-strict a desired default?

[shadowspawn]

I think the main argument for making strict off by default is to allow zero-config flags for quick/casual setup.

(The realisation that unknown arguments might be useful to detect comes later...)

[ljharb]

The easy thing should be the right thing; making the easy thing the wrong thing will cause problems later. Better to have a bit of friction in the beginning than a pile of tech debt later.

[bakkot]

I strongly agree with @ljharb. Not wanting validation (e.g. because you're doing it yourself in some way) is the unusual case.

There's also another reason to prefer strict-by-default: if a user is intuitively expecting strict behavior, they may well never realize that it's not strict (CLIs are often not very well tested, especially for malformed input). So they could easily end up shipping something which isn't strict, which would bite the consumers of their script. (I recently took over a library which had exactly that problem with nomnom, and will have to ship a breaking change to fix it.)

Conversely, if a user is expecting loose-by-default, it's presumably because they actually intend to pass input which would fail those checks. So they are much more likely to notice that their intuition is wrong, and realize they need to opt-in to skipping validation.

---

I think the main argument for making strict off by default is to allow zero-config flags for quick/casual setup.

Even in casual cases, the config necessary to specify flags (or to opt out of strict) is very small. While zero-config is a laudable goal, when it comes at significant risk of people shipping CLIs which still have sharp edges (namely, silently ignoring malformed input), I think the cost is not worth that benefit.

[shadowspawn]

The "Initial Proposal" just mentions unknown in two comments, one for and one against leaving it to author.

nodejs/node#35015

• util: add util.parseArgs() nodejs/node#35015 (comment)
• util: add util.parseArgs() nodejs/node#35015 (comment)

[bcoe]

I do not currently see a way of declaring an option as being known but not taking a value

@shadowspawn ☝️ this is what I don't like about strict by default for boolean options, we have to introduce a new configuration option solely to declare the options flag options you care about.

strict by default for options that takes arguments would only really apply to --foo=bar format arguments.

So, I'm not seeing how we would be strict by default without introducing quite a bit of ritual into how you configure your argument parser -- we need to add a config option that's purpose is "enumerate any keys that I care about that don't happen to have been defined in short, or withValue"...

Edit: one decision we could make that would perhaps make strict less weird, would be if we always require that all short options have a long form, and are therefore defined in short -- at which point we could check whether the argument is in --short or --withValue 🤔

[bakkot]

So, I'm not seeing how we would be strict by default without introducing quite a bit of ritual into how you configure your argument parser -- we need to add a config option that's purpose is "enumerate any keys that I care about that don't happen to have been defined in short, or withValue"...

Couldn't it just be an additional withoutValue key (or whatever) in the options object? That doesn't seem like "quite a bit of ritual", that's just... naming all of the things you're parsing. It's weirder not to have that: absent such a list, your options ends up naming only a subset of things you're looking for, with that subset depending on whether the argument happens to take a value. That would be very strange.

That is, the thing I'd expect, as a user, would be to do something along the lines of

util.parseArgs(argv, {
  withValue: ['some', 'options'],
  withoutValue: ['other', 'things'],
  // maybe also `short` and `multiples`, which would be required to be subsets of the above two lists
});

[shadowspawn]

Naming: Jordan Harband and I both thought of flags as not taking an argument. #12 (comment) #12 (comment)

For strict I would suggest listing options in flags and withValue and no free pass for mention in shorts and multiples.

It's weirder not to have that: absent such a list, your options ends up naming only a subset of things you're looking for, with that subset depending on whether the argument happens to take a value. That would be very strange.

For your interest and some context @bakkot, two of the four most downloaded argument parsers (npmcharts) default to zero-config parsing: yargs, and minimist. These two come from the same Optimist heritage, so not independent.

The other two of the top four are strict by default and do not support zero-config parsing as such: Commander and argparse. These would seem less strange to you. 😄

[bakkot]

Yup, I've looked at all of those before, and I do indeed find yargs and minimist to be extremely surprising. More to the point, I've regularly been bitten by applications built with those tools, because the author of the application failed to realize they needed to opt in to doing validation, which meant my typos were silently ignored. Which is a terrible experience all around.

Number of downloads isn't a great metric because often it just means there's a single popular package which depends on the package in question. argparse, for example, is used by js-yaml, which radically inflates argparse's download count. Compare, for example, command-line-args, which has significantly fewer downloads than argparse but also has more dependent packages listed on npm (and more stars on GitHub, though again not a great metric). command-line-args is also strict-by-default, incidentally.

[shadowspawn]

I've regularly been bitten by applications built with those tools, because the author of the application failed to realize they needed to opt in to doing validation, which meant my typos were silently ignored.

That is a good insight from experience, thanks.

Number of downloads isn't a great metric because often it just means there's a single popular package which depends on the package in question. argparse, for example, is used by js-yaml, which radically inflates argparse's download count.

I wanted to draw the line on the long-tail of arguments parsers somewhere, and had wondered why the other metrics for argparse were so much lower. Thanks. I might stop talking about the Big Four and stick to the Big Three as my standard references for consulting, and then a wide variety of other interesting packages...

[aaronccasanova]

Without member access to this repo I couldn't figure out an elegant way (with my fork) to branch off and PR against the proposed options API in PR #63, but I took a crack at extending it to support strict mode. With the restructured options API it was simple to identify unknown long options, short options, short option groups, and options with explicit values.

Here is the PR to Add strict mode to parser and if folks know how I can point this to PR #63 please let me know!

[bcoe]

Are people picturing that strict: true would throw an error object if an option that's not configured is provided, and strict: false would instead populate the error object on the object returned by the parse?

[shadowspawn]

To be useful for BYO validation and changing the rules, I think strict:false will need to return an array of errors? So caller can ignore the ones they don't care about, like say unrecognised options. (I was initially dubious about returning error as assumed parse result would be tainted by any error, but now thinking it might work for the errors we are likely to detect...)

I am imagining three modes of use:

• {strict:true}: use parseArgs default behaviour, client wraps in try/catch to display error.message more cleanly when they care
• {strict:false} with complete disregard for errors for quick prototyping.
• {strict:false} with careful treatment of errors to modify or extend behaviour

[ljharb]

I would expect strict true to throw errors, and strict false to build up an AggregateError of all errors that would have otherwise been thrown, and provide that on the result object somehow.

[bakkot]

It seems odd to create Errors and not throw them. If the user actually wanted those errors, they'd just use strict: true, surely?

I was expecting that setting strict to false would have it put unknown options within the results just like it does for known options (like was originally going to be the default behavior, as I understood it).

Another reasonable behavior would be to put the extra results on a separate object - so like let { options, positionals, unknownOptions } = parseArgs({ strict: false }), where unknownOptions has the same shape as options.

---

In any case, it would be good to have a concrete use case or use cases in mind for strict: false. @shadowspawn mentioned two: rapid prototypes and custom errors. For rapid prototypes, anything should work here as long as it's easy to get the values - so the Error design doesn't really work but either the mix-with-results design or the extra-object work fine. For custom errors, you want to know what the extra options were, presumably, for which the extra-object design seems nicest.

Do we expect that setting strict to false will disable validation other than that for unknown errors? What other validation is there which could in theory be disabled?

[ljharb]

Ignoring the errors is fine too, since someone who wants them can try/catch to get them (assuming we collect them all before throwing).

I do also agree that unknown options, in sloppy mode, should be available on the results object.

[shadowspawn]

@bakkot wrote:

I was expecting that setting strict to false would have it put unknown options within the results just like it does for known options (like was originally going to be the default behavior, as I understood it).

Correct. This is the original vision. Parse anything, zero-to-minimal config.

If client wants to do something different with the auto-discovered options, they can identify them by comparing against the input configuration.

[shadowspawn]

@bakkot wrote:

Do we expect that setting strict to false will disable validation other than that for unknown errors?

strict:false will not throw due to the contents of args. Best effort results.

What other validation is there which could in theory be disabled?

Now we get to the nitty-gritty, which we have been skirting around until now and waving the strict-will-cover-this wand. 😄

Possible strict errors:

• unknown option encountered
• option of type='string' used like a boolean option e.g. lone --string
• option of type='boolean' used like a string option e.g. --boolean=bogus
• group of short options with an option taking a value in the middle e.g. { args: '-abczdef', { options: z: { type='string' }, strict: true } This is parsable but pretty reasonable that strict says it is excessively dubious and difficult to read.

[shadowspawn]

@ljharb wrote:

Ignoring the errors is fine too, since someone who wants them can try/catch to get them (assuming we collect them all before throwing).

I don't understand this comment?

[ljharb]

@shadowspawn meaning, if we ignore the errors in sloppy mode, then someone who wants them can try/catch around a call in strict mode, and catch a thrown error to see what broke. Probably not the easiest, nicest, or fastest way to do it tho :-)

[shadowspawn]

Is anyone here a champion to drive errors in sloppy strict:false mode? Otherwise I suggest we focus on the cases we understand and want:

• throw error when strict:true
• best effort parse with no errors when strict:false, BYO validation

Enough for MVP?

[aaronccasanova]

@bakkot wrote:

What other validation is there which could in theory be disabled?

@shadowspawn wrote:

Possible strict errors:

• unknown option encountered
• option of type='string' used like a boolean option e.g. lone --string
• option of type='boolean' used like a string option e.g. --boolean=bogus
• group of short options with an option taking a value in the middle e.g. { args: '-abczdef', { options: z: { type='string' }, strict: true } This is parsable but pretty reasonable that strict says it is excessively dubious and difficult to read.

Subjective opinion, but I feel strict: true/MVP only needs to error on unknown options and backed by Darcy's initial comment:

The goal of this spec, which differentiated it from the nodejs/node#35015, was that you'd "bring your own validation" in reference to values & types.

I also think this is the cleanest way to introduce strict mode by avoiding littering the parser with strict checks and allowing us to slip in the unknown option validation in the store option util (as seen in my POC PR).

[aaronccasanova]

Is anyone here a champion to drive errors in sloppy strict:false mode?

I can give it a go after #63 is merged. I already have a WIP strict mode PR that builds off #63 that shouldn't be too difficult to add in some form of unknownOptions. Suggestions for the data structure and shape are welcome. Otherwise, I can play around with some ideas and look for feedback in the PR review.

Also, let me know if anyone else is interested in championing. I'm simply stepping up in the interest of adding support for a basic strict mode in the initial release (Node v18 🤞 )

[ljharb]

In strict mode, if I define an argument as a boolean and provide a value (--foo=bar, eg) then it should definitely throw.

[bakkot]

Yeah, I think that checking the values of options should be out of scope, but I feel like checking that --boolean flags are not used as if they take values should be in scope.

The philosophy I'd like to advocate here is that you can create a well-behaved* script with result = util.parseArgs({ options }), with no further validation required. To that end, parseArgs should enforce everything it allows the script author to specify. That way you as a script author can reason about the result of parseArgs: if the call didn't throw, you know that result conforms to options. People are going to assume that anyway, so if it doesn't enforce everything it allows you to specify then people are just going to end up with scripts which silently accept incorrect input. Which is bad.

*By "well-behaved" I mean that it will give the user helpful errors if they misuse it. Any script which does not do this is not well-behaved.

---

That said: what should be the behavior for such misused flags for strict: false? I feel like mixing non-boolean values for boolean flags (or conversely) in with the rest of the returned options is fraught even if you're explicitly opting in to non-strict behavior, which is perhaps a case for splitting out unknownOptions?

[shadowspawn]

That said: what should be the behavior for such misused flags for strict: false? I feel like mixing non-boolean values for boolean flags (or conversely) in with the rest of the returned options is fraught even if you're explicitly opting in to non-strict behavior, which is perhaps a case for splitting out unknownOptions?

I have thought a lot about what to include in the returned options. I am proposing (duck-typing):

If an option is used as a boolean, store result as for an option of type:boolean.
If an option is used with a value, store result as for an option of type:string.

The caller can easily check, no loss of information, and no new pattern in result.

References

• option of type='string' used like a boolean option e.g. lone --string
  • for long discussion, see Behaviour for withValue --foo followed by --bar ? #25
• option of type='boolean' used like a string option e.g. --boolean=bogus
  • for long discussion, see Behaviour for zero config --foo=a ? #24
• client detecting these errors (with various possible patterns of result)
  • Decide result conventions (includes detailed examples of client use) #38

[bakkot]

The caller can easily check, no loss of information, and no new pattern in result.

Well, they can, but will they? This is a persistent problem in JS: when the types are not as you expect, things end up giving the error at the wrong place.

How bad this is will depend on the results convention, but for simplicity I'm imagining something like what's proposed in that issue, where all the values for all options, boolean or string, end up in the same place. With such a design, if you write a script expecting --path=/a/b, but the user just writes --path part, the script is going to get { path: true }. If the script author forgets to check the type, and they do something like fetch(path), that's (hopefully!) going to fail with a message like "failed to fetch true". Which is not an especially useful error.

---

I think it's worth considering this with use cases in mind. For a rapid prototype or throwaway script, ending up with the wrong type for a configured option seems like it's just going to lead to more confusing errors, happening somewhat later in the execution. For authors hoping to provide custom errors, they now have to remember to check the typeof of every returned option. Neither of those seem like they're well served by mixing options-used-correctly with options-used-incorrectly.

What's the benefit of mixing the options-used-correctly with the options-used-incorrectly?

(Note that I'm only talking about the case where the script author does specify the option, but the user of the script misuses it, e.g. by passing a value to a boolean flag or failing to pass a value to a value-taking option.)

[shadowspawn]

Well, they can, but will they? This is a persistent problem in JS: when the types are not as you expect, things end up giving the error at the wrong place.

This wider issue and comments from you and @ljharb is why I think we are currently leaning towards making strict:true the default, although the original vision was definitely strict:false with BYO validation.

What's the benefit of mixing the options-used-correctly with the options-used-incorrectly?

A range of responses, see if any resonate. I don't think your suggestion is bad, but I don't currently think it is better.

1. The strict:false mode allows zero-config parsing. Using a variant of the minimist example, with proposed behaviours and results:

// console.log(parseArgs({ strict: false });
$ node example/parse.js -abc --beep=boop foo bar baz
{
  passedOptions: { a: true, b: true, c: true, beep: true },
  values: { a: true, b: true, c: true, beep: 'boop' }
  positionals: ['foo', 'bar', 'baz']
}

2. One way of looking at this is "the way they're stored matches the intention of the user, not the configurer, which will ensure the configurer can most accurately respond to the user's intentions."
   Behaviour for zero config --foo=a ? #24 (comment)

3. Counter question. If the author is not checking for errors, what's the benefit of quietly sticking some of the options somewhere else? This does not directly lead to better errors in the right place either.

[bakkot]

The strict:false mode allows zero-config parsing.

You can have zero-config parsing either way. With the design I'm proposing, the only difference is that you'd read from the things-which-don't-match-config property of the result rather than the things-which-match-config property.

the way they're stored matches the intention of the user, not the configurer, which will ensure the configurer can most accurately respond to the user's intentions

I feel like the configurer can respond just fine either way? The only difference is whether we make it clear where the user's intentions don't match the configurer's.

If the author is not checking for errors, what's the benefit of quietly sticking some of the options somewhere else? This does not directly lead to better errors in the right place either.

If the author isn't checking for errors, then it doesn't necessarily lead to better errors, but it probably leads to earlier errors: you'll usually get a message about something being missing or undefined rather than having the wrong value entirely. And you're less likely to end up doing something completely wrong, like writing to a file named "true".

Conversely, if the author is checking for errors, it's easier to do so when all of the things-which-don't-match-config are in the same place, separated from the things-which-do-match-config.

---

I'm not all that attached to the idea of splitting out unknown properties with strict: false, I should say. It seems like it helps, and does not hurt, both of the use cases we've identified, so I'm in favor of it, but strict: false is going to be require care to use anyway.

But I do feel strongly that strict: true should enforce (with an error) that options which are configured to take values are in fact given values.

[shadowspawn]

Thoughtful comments @bakkot thanks. I'm going to stop commenting on the idea of splitting out the un-authored properties to focus on the strict:true handling.

[shadowspawn]

(Explicitly adding my +1 to two of the cases I listed. 😄 )

• option of type='string' used like a boolean option e.g. lone --string
  • I strongly support this throwing an error when strict:true
• option of type='boolean' used like a string option e.g. --boolean=bogus
  • I strongly support this throwing an error when strict:true

The lack of detection of --boolean=bogus is one of the things that I think broke the original proposal. See for example: nodejs/node#35015 (comment)

@aaronccasanova wrote:

I also think this is the cleanest way to introduce strict mode by avoiding littering the parser with strict checks and allowing us to slip in the unknown option validation in the store option util

As for implementation, these two can both be detected in the same place as the unknown options in storeOptionValue, and won't litter the parser. (No slight intended, I care about that too.)

[shadowspawn]

group of short options with an option taking a value in the middle

Short version: in strict mode, this could throw same missing value error as for lone --string (identifying the short option which was expecting a value). I am +1 for that.

Long version

I was writing up a long story when I realised this case does not need to be identified to the user as a different error. Here is the journey...

1. I'll run through an example of the parsing that getopts and Commander do with this edge case. (I implementing the current Commander behaviour, so can speak to the reasoning if desired.)

If parseArgs does the same and does not generate an error in strict mode:

const result = parseArgs({
    args: ['-foo'],
    strict: true,
    options: { 
        f: { type: 'boolean' },
        o: { type: 'sting' }
    }
});
console.log(result.values);

{ f: true, o: 'o' }

2. If the implementation blindly expanded all middle shorts in a group as if they were boolean, we would effectively be parsing ['-f', '-o', '-o'] which would generate an error in strict mode due to the missing value for the first use of -o. I am ok with that outcome, but not with what the parsing would produce in strict:false mode.

However, this does suggest the strict:true mode does not need a different error, whatever the strict:false mode does. I like that... 💡

Implementation: this will require extra code, but I already had this possibility in mind in #68 and been thinking further about it since.

[shadowspawn]

🔨 Moving towards MVP Milestone 1 (#87)

I propose:

1. implement strict:true. Throw errors for:

• unknown option encountered
• option of type:'string' used like a boolean option e.g. lone --string
• option of type:'boolean' used like a string option e.g. --boolean=bogus

2. strict by default (!)
3. no error thrown or returned for strict:false, left entirely to author

---

Why default to strict? Some of the active people here are very keen and eloquent! No champions here for strict:false. Experience for (almost) zero-config is reasonable and visible opt-out:

const { values } = parseArgs({ strict: false });

Why not that error about short option group with string in middle? Rare. Get more important things done.

[ljharb]

In strict, what about multiple values for something configured not to be multiple?

[shadowspawn]

Not an error, last one wins.

[ljharb]

Seems like in strict mode that should be an error, no?

[bakkot]

No, definitely not. Arguments are always last-wins.

[shadowspawn]

(For interest, this was raised and discussed in previous PR to node: nodejs/node#35015 (comment) )

[shadowspawn]

Strict is indeed a goal, and enabled by default! Landed in #74.