-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wildcard regex token after literal token does not work #22
Comments
I presume you get the following error:
Parsus uses a traditional approach to parsing in general. It is split into two streams of work: tokenization and bottom-up parsing. The tokenization is the first step and it is done independently from the later parsing step. During tokenization (or lexing), a lexer takes all the tokens in the grammar (in their priority order) and tries to convert the raw stream of characters into a stream of tokens. This is done lazily, but the result is always deterministic, given the set of tokens. In your example, it works like this:
In short, the The workaround is to introduce an intermediate parser that tries both private val firstOrSecond by firstRegex or secondRegex
override val root by parser {
val first = firstRegex()
literal()
val second = firstOrSecond()
Pair(first.text, second.text)
} I would have to see if it makes sense to abandon the traditional tokenization approach to make such use-cases easier. Though, it's a deep topic. |
Hey @alllex, thank you for the fast response! Yes, the error I've got was along the lines of what you've presumed. Due to the explanation you gave, I can work with I've seen that you got started with #23 already, wow! From my understanding, this would change As this issue is mostly based on a misunderstanding of |
I read about
parsus
a while ago and wanted to incorporate it into a multi-platform side-project of mine.Sadly, I encountered the following behavior. I'm unsure whether I'm using the library wrong or I've encountered a bug, so any help is appreciated.
Basically I want to parse a string consisting of a limited character set, followed by a literal
:
and then ending in arbitrary text. My actual use case is a little more advanced, but this is the minimal subset which reproduces the problem.The same behavior can be observed, when using the combinator syntax.
Am I using parsus wrong? Or have I stumbled upon a bug?
The text was updated successfully, but these errors were encountered: