parsing python like languages

[schneidersoft]

How would I parse python like languages where indentation is used to handle scope?

[dolik-rce]

Hello @schneidersoft,

the indentation is just another character for the parser, so there is not much different to other parsers. You can also use references to make sure the indentation on each line is correct (at least in simple cases, more complex code would probably need to check this in code).

Here is a very simple grammar to parse python-like function definition:

function <- "def " identifier "():\n" <indent> statement "\n" ($1 statement "\n")*

identifier <- [_a-zA-Z][_a-zA-Z0-9]*

indent <- " "+ / "\t"+

statement <- [^ \t\n][^\n]*

It would report syntax error if you run it on incorrectly indented code.

[dolik-rce]

PS: Here is a full python grammar (just in slightly different format than the one used by PackCC), if you need some inspiration: https://docs.python.org/3/reference/grammar.html

[schneidersoft]

Right. I was wondering how to integrate a tokenizer that would produce the INDENT end DEDENT tokens packcc would then be able to use.

[mingodad]

Look here https://github.com/exaloop/codon/blob/develop/codon/parser/peg/grammar.peg how it uses https://github.com/yhirose/cpp-peglib to parse a Python like language.