Investigate Applications of deeper RegExp analysis

[bd82]

There seems more and more issues that can could be better resolved if Chevrotain could
perform deep analysis of RegExp patterns.

• Identifying patterns that could contain line terminators automatically instead of forcing users
  to specify those explicitly using the line_breaks flag, see details.

• Correctly identifying usage of (disallowed) start/end anchors in regExps, see details.

• Possible performance improvements by compiling simple RegExps to state machines using
  CharCodeAt.

  • Mainly relevant where content security policies do not prevent eval/ Function constructor.

• Possible performance optimization to create a large switch case which tests the current charCode
  to reduce the number of possible regExps to be checked.

  • The greater the number of types of tokens the greater the benefit.

[bd82]

@RReverser wrote:

but you can just use some existing regexp parser like https://github.com/DmitrySoshnikov/regexp-tree to isolate changes, and perform analysis only on the interesting bits.

[bd82]

Performance optimizations using the first charCode to filter out irrelevant patterns
provided a large boost.

#682

[bd82]

#709

[bd82]

#710

[bd82]

Issues were open for each subtask.