You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would like to parse the following (which is a simplified example of the full text I would like to parse).
Version: 1.2.3
Features:
Fixes:
Following Version is a list of category strings. Each category must prefixed with two spaces and suffixed with a colon :.
I'd like to parse it into this class:
data classDemo(
valversion:String,
valcategories:List<String>
)
I have written a parser (see full code below) that takes the leading whitespace into account.
/** leading category-name whitespace, to be ignored */privateval categoryNameIndent by literalToken("")
privateval categoryNameSuffix by literalToken(":")
privateval categoryName by -categoryNameIndent * text *-categoryNameSuffix
This error is very confusing because it seems to have swapped around the expected/found. I didn't expect EOF, while I did expect LiteralToken(' '). And even then, why did the parser not find the literal token? It's hard to figure out, even when debugging, so help would be appreciated.
funmain() {
val demo =DemoGrammar.parseEntire(
/* language=text */"""Version: 1.2.3 Features: Fixes:""".trimIndent()
)
println("parsed demo: $demo")
}
object DemoGrammar : Grammar<Demo>(debugMode = true) {
privateval newline by regexToken("""\n|\r\n|\r""")
privateval text by regexToken(""".+""")
privateval versionTag by literalToken("Version: ")
privateval version by -versionTag * text
privateval categoryNameIndent by literalToken("")
privateval categoryNameSuffix by literalToken(":")
privateval categoryName by -categoryNameIndent * text *-categoryNameSuffix
privateval categorySection:Parser<String> by parser {
println("parsing CategorySection")
val name = categoryName().text
println(" name: $name")
println(newline())
name
}
overrideval root:Parser<Demo> by parser {
val version = version().text
println(" version:$version")
val categories = repeatZeroOrMore(categorySection)
repeatZeroOrMore(newline)
Demo(
version = version,
categories = categories,
)
}
}
The text was updated successfully, but these errors were encountered:
aSemy
changed the title
MismatchedToken error: Token(EOF) is expected, despite requesting actual token of LiteralToken(' ')
MismatchedToken error: Token(EOF) is expected, despite requesting the found token of LiteralToken(' ')Jun 16, 2023
This error message can indeed be cryptic, but in general, it would not be possible to improve it significantly. The "mismatched token" is a last resort kind of error, when the parser ran out of alternatives. Still, something could be improved, and I'll consider trying something out in this area.
Regarding your grammar, I think the main issue is the moment you expect a newline. In the grammar you provided, you try parsing things in the following order:
version
(then you are not trying to parse a newline)
zero or more of
category name (that starts with indent)
The grammar misbehaves on your input, because there is a newline immediately following the version. So, you either need to add parsing of a newline inside the version parser, which I don't recommend. Or, you can say that each category must start with at least one newline, which I would recommend.
privateval categorySection:Parser<String> by parser {
println("parsing CategorySection")
repeatOneOrMore(newline) // <---------------- parsing at least one newlineval name = categoryName().text
println(" name: $name")
name
}
There is another problem with your grammar that is not obvious because of a bug in Parsus.
You declare a wildcard regex .+ before you declare literal tokens, such as categoryNameIndent or categoryNameSuffix. Parsus is sensitive to the declaration order of tokens. It would try to parse the tokens in this order, stopping as soon as it finds a token that matches. Due to a bug, the regex tokens lose their declaration order priority. This will be fixed in the next release.
What you need to do to protect from the bug: move the text literal declaration below all other token declarations. So, for your grammar, move it after categoryNameSuffix.
I would like to parse the following (which is a simplified example of the full text I would like to parse).
Following
and suffixed with a colon
Version
is a list of category strings. Each category must prefixed with two spaces:
.I'd like to parse it into this class:
I have written a parser (see full code below) that takes the leading whitespace into account.
However, I get an error
This error is very confusing because it seems to have swapped around the expected/found. I didn't expect EOF, while I did expect
token? It's hard to figure out, even when debugging, so help would be appreciated.
LiteralToken(' ')
. And even then, why did the parser not find the literalOutput:
The text was updated successfully, but these errors were encountered: