Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

util: add util.parseArgs() #35015

Closed
wants to merge 14 commits into from
160 changes: 160 additions & 0 deletions doc/api/util.md
Original file line number Diff line number Diff line change
Expand Up @@ -946,6 +946,161 @@ Otherwise, returns `false`.
See [`assert.deepStrictEqual()`][] for more information about deep strict
equality.

## `util.parseArgs([argv][, options])`
<!-- YAML
added: REPLACEME
-->

* `argv` {string[]|Object} (Optional) Array of argument strings; defaults
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:

Suggested change
* `argv` {string[]|Object} (Optional) Array of argument strings; defaults
* `argv` {string[]} (Optional) Array of argument strings; defaults

since this argument cannot be an object itself, it is only an object if it is options. I don't think that we include 'replacement/nested/optional' types in our docs in such cases. IMO it will only confuse the reader as it removes clear type-boundary between optional arguments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would prefer to leave as-is unless the convention is explicitly documented somewhere. afaik doing it the suggested way does not work very well if your docstrings are read by machines (e.g., TS language server), which is why I wrote it the way I did

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to keep this only being an array of strings. I do not understand the need for supporting anything that is not an array of strings given that process.argv is an array of strings. Can you please articulate why this is needed?


From experience adding types overloads are a cause issues down the line with the Buffer constructor being the most notable case of this. This is also very present in the streams API and other places, and I would define most of them as problematic one way or another.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mcollina It allows someone to supply an explicit array of arguments, or just an options object, where the value would default to process.argv. we could make it so you must pass process.argv, but that is likely boilerplate in 90% of cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw this comment is in regards to a docstring, unless you are talking about something else

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I misunderstood this. Object should be removed from this line as it is confusly. We normally mark this parameter as optional and that's it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still needs to be fixed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems resolved? Can we mark as resolved?

to [`process.argv.slice(2)`](process_argv). If an Object, the default is used,
and this parameter is considered to be the `options` parameter.
* `options` {Object} (Optional) The `options` parameter is an
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The word "option" has three different meanings in the added documentation: (1) JavaScript options passed via an options object, (2) arguments with a value, (3) any argument that starts with a -. I can see this leading to confusion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly we say "option" a lot, but I am unsure how to reconcile this given the word "option" is used in both JS and CLI contexts to mean entirely separate things. Even if we adopted the IEEE standard terminology, we'd still have that problem.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not be more verbose and say "command line parameter(s)"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there's any clear & obvious change to be made here. if someone thinks of one, a future PR can address it, but IMO this is good enough for now

object supporting the following properties:
* `optionsWithValue` {string[]|string} (Optional) One or more argument
boneskull marked this conversation as resolved.
Show resolved Hide resolved
strings which _expect a value_ when present in `argv` (see [Options][]
for details)
* `multiOptions` {string[]|string} (Optional) One or more argument
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to conflict with the definition of "Options" below, given that flags aren't options by these definitions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Maybe changing this name to multiple would make more sense?

strings which, when appearing multiple times in `argv`, will be concatenated
into an Array
* Returns: {Object} An object having properties:
* `options` {Object}, having properties and values corresponding to parsed
[Options][] and [Flags][]
* `positionals` {string[]}, containing [Positionals][]

The `util.parseArgs` function parses command-line arguments from an Array of
strings and returns an object representation.

Example using [`process.argv`][]:

```js
// script.js
// called via `node script.js --foo bar baz`
const argv = util.parseArgs();

if (argv.foo === true) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

argv.options.foo?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

console.log(argv.positionals); // prints [ 'bar', 'baz' ]
}
```

Example using a custom `argv` and the `optionsWithValue` option:

```js
const argv = util.parseArgs(
['--foo', 'bar', 'baz'],
{ optionsWithValue: ['foo'] }
);

// argv.foo === 'bar'
if (argv.foo === 'bar') {
console.log(argv.positionals); // prints [ 'baz' ]
}
```

Example using custom `argv`, `optionsWithValue`, and the `multiOptions` option:

```js
const argv = util.parseArgs(
['--foo', 'bar', '--foo', 'baz'],
{ optionsWithValue: 'foo', multiOptions: 'foo' }
);

console.log(argv.options.bar); // prints [ 'bar', 'baz' ]
```

Example with custom `argv` and `multiOptions`:

```js
const argv = util.parseArgs(
['-v', '-v', '-v'],
{ multiOptions: 'v' }
);

console.log(argv.options.v); // prints [ true, true, true ]
```

[`ERR_INVALID_ARG_TYPE`][] will be thrown if the `argv` parameter is not an
Array.

Arguments fall into one of three catgories:

### Flags

_Flags_ are arguments which begin with one or more dashes (`-`), and _do not_
have an associated string value (e.g., `node app.js --verbose`).

* These will be parsed automatically; you do not need to "declare" them
boneskull marked this conversation as resolved.
Show resolved Hide resolved
* The Flag _name_ is the string following the prefix of _one or more_ dashes,
e.g., the name of `--foo` is `foo`
* Flag names become property names in the `options` property of the returned
object
* By default, when appearing any number of times in the `argv` Array, the value
of the property will be `true`. To get a "count" of the times a Flag is
repeated, specify the Flag name in the `multiOptions` option; this will parsed
boneskull marked this conversation as resolved.
Show resolved Hide resolved
to an Array of `true` values, and you can derive the "count" from the `length`
property of this Array
* When a Flag appears in `multiOptions`, and when provided in `argv`, the value
in the returned object will _always_ be an Array (even if it is only provided
once)
* A Flag appearing in `multiOptions` but not in the `argv` Array will be omitted
from the `options` property of the returned object
* If a string value is erroneously provided in `argv` for a Flag via the `=`
separator, the string value will be replaced with `true`; e.g.,
`['--require=script.js']` becomes `{options: {require: true}}, positionals:
[]}`
boneskull marked this conversation as resolved.
Show resolved Hide resolved

### Options

_Options_ are arguments which begin with one or more dashes (`-`), and _do_ have
an associated value (e.g., `node app.js --require script.js`).

* Use the `optionsWithValue` option to `util.parseArgs` to declare Options
* The Option _name_ is the string following the prefix of one-or-more dashes,
e.g., the name of `--foo` is `foo`
* The Option _value_ is the next string following the name, e.g., the Option
value of `['--foo' 'bar']` is `bar`
* Option values may be provided _with or without_ a `=` separator (e.g.,
`['--require=script.js']` is equivalent to `['--require', 'script.js']`)
* If an Option value is not provided, the Option will be omitted from the
`options` property of the returned object
* An argument-like value (a value beginning with one or more dashes) immediately
following an Option in the `argv` Array will cause the Option to be omitted
from the `options` property of the returned object _unless_ the `=` separator
is used; e.g., `['--foo', '--bar']` where `foo` is an Option will return
`{options: {bar: true}, positionals: []}`, but `['--foo=--bar']` will return
`{options: {foo: '--bar'}, positionals: []}`
* When an Option name appears in the Array (or string) of `optionsWithValue`,
and does _not_ appear in the `argv` Array, the resulting object _will not_
contain a property with this Option name (e.g., `util.parseArgs(['--bar'], {
optionsWithValue: 'foo' })` will result in `{options: {bar: true},
positionals: [] }`
* When an Option appears in `multiOptions`, and when provided in `argv`, the
value in the returned object will _always_ be an Array (even if it is only
provided once)

### Positionals

_Positionals_ (or "positional arguments") are arguments which _do not_ begin
with one or more dashes (e.g., `['script.js']`), _and/or_ all items in the
`argv` Array following a `--` (e.g., `['--', 'script.js']`).

* Positionals appear in the Array property `positionals` of the returned object
* The `positionals` property will _always_ be present and an Array, even if
empty
* If present in the `argv` Array, `--` is discarded and is omitted from the
returned object
* Positionals will _always_ be parsed verbatim (e.g., `['--', '--foo']` will
result in an object of `{positionals: ['--foo'], options: {}}`)

Please note:
boneskull marked this conversation as resolved.
Show resolved Hide resolved

* `util.parseArgs` does not consider "short" arguments (e.g., `-v`) to be
different than "long" arguments (e.g., `--verbose`). Furthermore, it does not
allow concatenation of short arguments (e.g., `-v -D` cannot be expressed as
`-vD`).
* _No_ conversion to or from "camelCase" occurs; a Flag or Option name of
`no-color` results in an object with a `no-color` property. A Flag or Option
name of `noColor` results in an object with a `noColor` property.

## `util.promisify(original)`
<!-- YAML
added: v8.0.0
Expand Down Expand Up @@ -2497,8 +2652,13 @@ util.log('Timestamped message.');
[compare function]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#Parameters
[constructor]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/constructor
[default sort]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort
[`ERR_INVALID_ARG_TYPE`]: errors.html#ERR_INVALID_ARG_TYPE
[Flags]: #util_flags
[global symbol registry]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol/for
[list of deprecated APIS]: deprecations.html#deprecations_list_of_deprecated_apis
[`napi_create_external()`]: n-api.html#n_api_napi_create_external
[Options]: #util_options
[Positionals]: #util_positionals
[`process.argv`]: process.html#process_process_argv
[semantically incompatible]: https://github.com/nodejs/node/issues/4179
[util.inspect.custom]: #util_util_inspect_custom
145 changes: 145 additions & 0 deletions lib/internal/util/parse_args.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
'use strict';

const {
ArrayIsArray,
ArrayPrototypePush,
ArrayPrototypeSlice,
SafeSet,
StringPrototypeReplace,
StringPrototypeSplit,
StringPrototypeStartsWith,
} = primordials;
const { ERR_INVALID_ARG_TYPE } = require('internal/errors').codes;

/**
* Returns an Object representation of command-line arguments.
*
* Default behavior:
* - All arguments are considered "boolean flags"; in the `options` property of
* returned object, the key is the argument (if present), and the value will
* be `true`.
* - Uses `process.argv.slice(2)`, but can accept an explicit array of strings.
* - Argument(s) specified in `opts.optionsWithValue` will have `string` values
* instead of `true`; the subsequent item in the `argv` array will be consumed
* if a `=` is not used
* - "Bare" arguments (positionals) are those which do not begin with a `-` or
* `--` and those after a bare `--`; these will be returned as items of the
* `positionals` array
* - The `positionals` array will always be present, even if empty.
* - The `options` Object will always be present, even if empty.
* @param {string[]} [argv=process.argv.slice(2)] - Array of script arguments as
* strings
* @param {Object} [options] - Options
* @param {string[]|string} [opts.optionsWithValue] - This argument (or
* arguments) expect a value
* @param {string[]|string} [opts.multiOptions] - This argument (or arguments)
boneskull marked this conversation as resolved.
Show resolved Hide resolved
* can be specified multiple times and its values will be concatenated into an
* array
* @returns {{options: Object, positionals: string[]}} Parsed arguments
* @example
* parseArgs(['--foo', '--bar'])
* // {options: { foo: true, bar: true }, positionals: []}
* parseArgs(['--foo', '-b'])
* // {options: { foo: true, b: true }, positionals: []}
* parseArgs(['---foo'])
* // {options: { foo: true }, positionals: []}
* parseArgs(['--foo=bar'])
* // {options: { foo: true }, positionals: []}
* parseArgs([--foo', 'bar'])
* // {options: {foo: true}, positionals: ['bar']}
* parseArgs(['--foo', 'bar'], {optionsWithValue: 'foo'})
* // {options: {foo: 'bar'}, positionals: []}
* parseArgs(['foo'])
* // {options: {}, positionals: ['foo']}
* parseArgs(['--foo', '--', '--bar'])
* // {options: {foo: true}, positionals: ['--bar']}
* parseArgs(['--foo=bar', '--foo=baz'])
* // {options: {foo: true}, positionals: []}
* parseArgs(['--foo=bar', '--foo=baz'], {optionsWithValue: 'foo'})
* // {options: {foo: 'baz'}, positionals: []}
* parseArgs(['--foo=bar', '--foo=baz'], {
* optionsWithValue: 'foo', multiOptions: 'foo'
* }) // {options: {foo: ['bar', 'baz']}, positionals: []}
* parseArgs(['--foo', '--foo'])
* // {options: {foo: true}, positionals: []}
* parseArgs(['--foo', '--foo'], {multiOptions: ['foo']})
* // {options: {foo: [true, true]}, positionals: []}
* parseArgs(['--very-wordy-option'])
* // {options: {'very-wordy-option': true}, positionals: []}
*/
const parseArgs = (
argv = ArrayPrototypeSlice(process.argv, 2),
options = { optionsWithValue: [] }
) => {
if (!ArrayIsArray(argv)) {
options = argv;
argv = ArrayPrototypeSlice(process.argv, 2);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defaulting to process.argv.slice(2) is fine but I think the implementation should somewhat also consider the eval usecase, right now if I try running node -p|-e I get a very unhelpful error:

$ node -p 'require('util').parseArgs()' foo bar
internal/validators.js:122
    throw new ERR_INVALID_ARG_TYPE(name, 'string', value);
    ^

TypeError [ERR_INVALID_ARG_TYPE]: The "id" argument must be of type string. Received an instance of Object
    at new NodeError (internal/errors.js:253:15)
    at validateString (internal/validators.js:122:11)
    at Module.require (internal/modules/cjs/loader.js:972:3)
    at require (internal/modules/cjs/helpers.js:88:18)
    at [eval]:1:1
    at Script.runInThisContext (vm.js:132:18)
    at Object.runInThisContext (vm.js:309:38)
    at internal/process/execution.js:77:19
    at [eval]-wrapper:6:22
    at evalScript (internal/process/execution.js:76:60) {
  code: 'ERR_INVALID_ARG_TYPE'
}

In the past I've done something like process.argv.slice(require.main ? 2 : 1) in order to support it (though there might be better ways to handle the check in core).

IMO parseArgs should either handle eval/script properly OR at least throw an useful error instead 😊

}
if (typeof options !== 'object' || options === null) {
throw new ERR_INVALID_ARG_TYPE(
'options',
'object',
options);
}
if (typeof options.optionsWithValue === 'string') {
options.optionsWithValue = [options.optionsWithValue];
}
if (typeof options.multiOptions === 'string') {
options.multiOptions = [options.multiOptions];
}
Comment on lines +88 to +93
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my request for changes still stands on these

const optionsWithValue = new SafeSet(options.optionsWithValue || []);
const multiOptions = new SafeSet(options.multiOptions || []);
const result = { positionals: [], options: {} };

let pos = 0;
while (true) {
const arg = argv[pos];
if (arg === undefined) {
boneskull marked this conversation as resolved.
Show resolved Hide resolved
return result;
}
boneskull marked this conversation as resolved.
Show resolved Hide resolved
if (StringPrototypeStartsWith(arg, '-')) {
// Everything after a bare '--' is considered a positional argument
// and is returned verbatim
if (arg === '--') {
ArrayPrototypePush(
result.positionals, ...ArrayPrototypeSlice(argv, ++pos)
);
return result;
}
// Any number of leading dashes are allowed
const argParts = StringPrototypeSplit(StringPrototypeReplace(arg, /^-+/, ''), '=');
const optionName = argParts[0];
let optionValue = argParts[1];

// Consume the next item in the array if `=` was not used
// and the next item is not itself a flag or option
if (optionsWithValue.has(optionName)) {
if (optionValue === undefined) {
optionValue = StringPrototypeStartsWith(argv[pos + 1], '-') ||
boneskull marked this conversation as resolved.
Show resolved Hide resolved
argv[++pos];
}
} else {
optionValue = true;
}

if (multiOptions.has(optionName)) {
// Consume the next item in the array if `=` was not used
// and the next item is not itself a flag or option
if (result.options[optionName] === undefined) {
result.options[optionName] = [optionValue];
} else {
ArrayPrototypePush(result.options[optionName], optionValue);
}
} else {
result.options[optionName] = optionValue;
}
} else {
ArrayPrototypePush(result.positionals, arg);
}
pos++;
}
};

module.exports = {
parseArgs
};
2 changes: 2 additions & 0 deletions lib/util.js
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ const { validateNumber } = require('internal/validators');
const { TextDecoder, TextEncoder } = require('internal/encoding');
const { isBuffer } = require('buffer').Buffer;
const types = require('internal/util/types');
const { parseArgs } = require('internal/util/parse_args');

const {
deprecate,
Expand Down Expand Up @@ -270,6 +271,7 @@ module.exports = {
isFunction,
isPrimitive,
log,
parseArgs,
promisify,
TextDecoder,
TextEncoder,
Expand Down
1 change: 1 addition & 0 deletions node.gyp
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,7 @@
'lib/internal/util/debuglog.js',
'lib/internal/util/inspect.js',
'lib/internal/util/inspector.js',
'lib/internal/util/parse_args.js',
'lib/internal/util/types.js',
'lib/internal/http2/core.js',
'lib/internal/http2/compat.js',
Expand Down
1 change: 1 addition & 0 deletions test/parallel/test-bootstrap-modules.js
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ const expectedModules = new Set([
'NativeModule internal/util',
'NativeModule internal/util/debuglog',
'NativeModule internal/util/inspect',
'NativeModule internal/util/parse_args',
'NativeModule internal/util/types',
'NativeModule internal/validators',
'NativeModule internal/vm/module',
Expand Down
Loading