Skip to content

Commit

Permalink
Reserved: Improve error when reserved word used as a label name and a…
Browse files Browse the repository at this point in the history
…dd documentation for plugins API

Before this commit error looks like (for input `start = break:'a'`)

> Expected "!", "$", "&", "(", "*", "+", ".", "/", "/*", "//", ";", "?", character class, code block, comment, end of line, identifier, literal, or whitespace but ":" found.

After this error looks like

> Expected identifier but reserved word "break" found.
  • Loading branch information
Mingun committed May 23, 2021
1 parent cc04fa0 commit 7f20170
Show file tree
Hide file tree
Showing 4 changed files with 134 additions and 6 deletions.
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,14 @@ Released: TBD
- `zero_or_more`
- `one_or_more`
- `group`
- Add a new option `config.reservedWords: string[]`, avalible for plugins in their
`use()` method. Using this option, a plugin can change the list of words that
cannot be used.

By default this new option contains an array with [reserved JavaScript words][reserved]
[@Mingun](https://github.com/peggyjs/peggy/pull/150)

[reserved]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Lexical_grammar#reserved_keywords_as_of_ecmascript_2015

### Bug fixes

Expand Down
46 changes: 43 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ object to `peg.generate`. The following options are supported:
- `output` — if set to `"parser"`, the method will return generated parser
object; if set to `"source"`, it will return parser source code as a string
(default: `"parser"`)
- `plugins` — plugins to use
- `plugins` — plugins to use. See the [Plugins API](#plugins-api) section
- `trace` — makes the parser trace its progress (default: `false`)

## Using the Parser
Expand Down Expand Up @@ -453,15 +453,20 @@ If you need to return the matched text in an action, use the
#### _label_ : _expression_

Match the expression and remember its match result under given label. The label
must be a JavaScript identifier.
must be a JavaScript identifier, but not in the list of reserved words.
By default this is a list of [JavaScript reserved words][reserved],
but [plugins](#plugins-api) can change it.

Labeled expressions are useful together with actions, where saved match results
can be accessed by action's JavaScript code.

#### _@_ ( _label_ : )? _expression_

Match the expression and if the label exists, remember its match result under
given label. The label must be a JavaScript identifier if it exists.
given label. The label must be a JavaScript identifier if it exists, but not
in the list of reserved words. By default this is a list of
[JavaScript reserved words][reserved], but [plugins](#plugins-api) can
change it.

Return the value of this expression from the rule, or "pluck" it. You may not
have an action for this rule. The expression must not be a semantic predicate
Expand Down Expand Up @@ -721,6 +726,39 @@ Discussions page][unicode].
[BMP]: https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane
[unicode]: https://github.com/peggyjs/peggy/discussions/15

## Plugins API

A plugin is an object with the `use(config, options)` method. That method will be
called for all plugins in the `options.plugins` array, supplied to the `generate()`
method.

`use` accepts those parameters:
- `config` — object with the following properties:
- `parser``Parser` object, by default the `peggy.parser` instance. That object
will be used to parse the grammar. Plugin can replace this object
- `passes` — mapping `{ [stage: string]: Pass[] }` that represents compilation
stages that would applied to the AST, returned by the `parser` object. That
mapping will contain at least the following keys:
- `check` — passes that check AST for correctness. They shouldn't change the AST
- `transform` — passes that performs various optimizations. They can change
the AST, add or remove nodes or their properties
- `generate` — passes used for actual code generating

Plugin that implement a pass usually should push it to the end of one of that
arrays. Pass is a simple function with signature `pass(ast, options)`:
- `ast` — the AST created by the `config.parser.parse()` method
- `options` — compilation options passed to the `peggy.compiler.compile()` method.
If parser generation is started because `generate()` function was called that
is also an options, passed to the `generate()` method
- `reservedWords` — string array with a list of words that shouldn't be used as
label names. This list can be modified by plugins. That property is not required
to be sorted or not contain duplicates, but it is recommend to remove duplicates.

Default list contains [JavaScript reserved words][reserved], and can be found
in the `peggy.RESERVED_WORDS` property.
- `options` — build options passed to the `generate()` method. A best practice for
a plugin would look for its own options under a `<plugin_name>` key.

## Compatibility

Both the parser generator and generated parsers should run well in the following
Expand Down Expand Up @@ -749,3 +787,5 @@ Peggy was originally developed by [David Majda](https://majda.cz/)
You are welcome to contribute code. Unless your contribution is really trivial
you should [get in touch with us](https://github.com/peggyjs/peggy/discussions)
first — this can prevent wasted effort on both sides.

[reserved]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Lexical_grammar#reserved_keywords_as_of_ecmascript_2015
66 changes: 63 additions & 3 deletions docs/documentation.html
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ <h2 id="table-of-contents">Table of Contents</h2>
</li>
<li><a href="#error-messages">Error Messages</a></li>
<li><a href="#locations">Locations</a></li>
<li><a href="#plugins-api">Plugins API</a></li>
<li><a href="#compatibility">Compatibility</a></li>
</ul>

Expand Down Expand Up @@ -246,7 +247,7 @@ <h3 id="generating-a-parser-javascript-api">JavaScript API</h3>
a string (default: <code>"parser"</code>).</dd>

<dt><code>plugins</code></dt>
<dd>Plugins to use.</dd>
<dd>Plugins to use. See the [Plugins API](#plugins-api) section.</dd>

<dt><code>trace</code></dt>
<dd>Makes the parser trace its progress (default: <code>false</code>).</dd>
Expand Down Expand Up @@ -565,7 +566,9 @@ <h3 id="grammar-syntax-and-semantics-parsing-expression-types">Parsing Expressio

<dd>
<p>Match the expression and remember its match result under given label.
The label must be a JavaScript identifier.</p>
The label must be a JavaScript identifier, but not in the list of reserved words.
By default this is a list of <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Lexical_grammar#reserved_keywords_as_of_ecmascript_2015">JavaScript reserved words</a>,
but <a href="#plugins-api">plugins</a> can change it.</p>

<p>Labeled expressions are useful together with actions, where saved match
results can be accessed by action's JavaScript code.</p>
Expand All @@ -576,7 +579,9 @@ <h3 id="grammar-syntax-and-semantics-parsing-expression-types">Parsing Expressio
<dd>
<p>Match the expression and if the label exists, remember its match result
under given label. The label must be a JavaScript identifier if it
exists.</p>
exists, but not in the list of reserved words.
By default this is a list of <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Lexical_grammar#reserved_keywords_as_of_ecmascript_2015">JavaScript reserved words</a>,
but <a href="#plugins-api">plugins</a> can change it.</p>

<p>Return the value of this expression from the rule, or "pluck" it. You
may not have an action for this rule. The expression must not be a
Expand Down Expand Up @@ -807,6 +812,61 @@ <h2 id="locations">Locations</h2>
<p>Changing this behavior may be a breaking change and will not to be done before
Peggy 2.0. You can join to the discussion for this topic on the <a href="https://github.com/peggyjs/peggy/discussions/15">GitHub Discussions page</a>.</p>

<h2 id="plugins-api">Plugins API</h2>

<p>A plugin is an object with the <code>use(config, options)</code> method. That method will be
called for all plugins in the <code>options.plugins</code> array, supplied to the <code>generate()</code>
method.</p>

<p><code>use</code> accepts those parameters:</p>

<h3><code>config</code></h3>
<p>Object with the following properties:</p>

<dl>
<dt><code>parser</code></dt>
<dd><code>Parser</code> object, by default the <code>peggy.parser</code> instance. That object
will be used to parse the grammar. Plugin can replace this object</dd>

<dt><code>passes</code></dt>
<dd>
<p>Mapping <code>{ [stage: string]: Pass[] }</code> that represents compilation
stages that would applied to the AST, returned by the <code>parser</code> object. That
mapping will contain at least the following keys:</p>

<ul>
<li><code>check</code> — passes that check AST for correctness. They shouldn't change the AST</li>
<li><code>transform</code> — passes that performs various optimizations. They can change
the AST, add or remove nodes or their properties</li>
<li><code>generate</code> — passes used for actual code generating</li>
</ul>

<p>Plugin that implement a pass usually should push it to the end of one of that
arrays. Pass is a simple function with signature <code>pass(ast, options)</code>:</p>

<ul>
<li><code>ast</code> — the AST created by the <code>config.parser.parse()</code> method</li>
<li><code>options</code> — compilation options passed to the <code>peggy.compiler.compile()</code> method.
If parser generation is started because <code>generate()</code> function was called that
is also an options, passed to the <code>generate()</code> method</li>
</ul>
</dd>

<dt><code>reservedWords</code></dt>
<dd>
<p>String array with a list of words that shouldn't be used as
label names. This list can be modified by plugins. That property is not required
to be sorted or not contain duplicates, but it is recommend to remove duplicates.</p>

<p>Default list contains <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Lexical_grammar#reserved_keywords_as_of_ecmascript_2015">JavaScript reserved words</a>, and can be found
in the <code>peggy.RESERVED_WORDS</code> property.</p>
</dd>
</dl>

<h3><code>options</code></h3>
<dd>Build options passed to the <code>generate()</code> method. A best practice for
a plugin would look for its own options under a <code>&lt;plugin_name&gt;</code> key.</dd>

<h2 id="compatibility">Compatibility</h2>

<p>Both the parser generator and generated parsers should run well in the
Expand Down
20 changes: 20 additions & 0 deletions test/api/plugin-api.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,11 @@ describe("plugin API", function() {
config.passes.generate.forEach(pass => {
expect(pass).to.be.a("function");
});

expect(config.reservedWords).to.be.an("array");
config.reservedWords.forEach(word => {
expect(word).to.be.a("string");
});
}
};

Expand Down Expand Up @@ -104,6 +109,21 @@ describe("plugin API", function() {
expect(parser.parse("a")).to.equal(42);
});

it("can change list of reserved words", function() {
const plugin = {
use(config) {
config.reservedWords = [];
}
};

expect(() => {
peg.generate(
"start = " + peg.RESERVED_WORDS[0] + ":'a'",
{ plugins: [plugin], output: "source" }
);
}).to.not.throw();
});

it("can change options", function() {
const grammar = [
"a = 'x'",
Expand Down

0 comments on commit 7f20170

Please sign in to comment.