Minor perf improvements and code touch ups. #222
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, a byte based regex would always use byte based instructions. This isn't actually necessary. If the regex has the Unicode flag enabled, for example, then Unicode instructions can and should be used. We can't mix and match though. If any part of the regex could match invalid UTF-8, then we need to drop back to byte-based matching. We could be more aggressive in the parser. In particular, we could check if arbitrary character classes only match UTF-8, and if so, represent them as Unicode classes instead of byte classes. This optimization is only applicable to the NFA engines, since the DFA operates exclusively on bytes. In particular, the NFA engines are much faster with Unicode instructions.
It has exactly the same information and therefore was purely redundant. We now uses states themselves as keys in the compiled map.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
bytes::Regexmore aggressively. We could do more on this front. Unicode instructions are quite a bit faster for the NFA engines to execute.StateKeytype, since it is exactly equivalent toState.