Fix memory leak while parsing improperly terminated inherit-expressions

[Ma27]

First of all, big thanks to @fufexan who helped me to reliably reproduce
this.

Originally discovered in rnix-lsp[1], but I confirmed that
nixpkgs-fmt is also affected.

Basically, when having an expression such as

let
  inherit

the parser would wait for a TOKEN_SEMICOLON indefinitely. The actual
problem however is that self.parse_val() always detects the SAME
syntax-error, i.e. "unexpected EOF". This will be written indefinetely
into self.errors. However, errors is of type Vec<ParseError> and a
vector in Rust grows in an amortized fashion[2] which means that if an
entry is pushed and the vector exceeds the currently allocated size, it
will be ~doubled (though the exact growth-factor isn't constant).

This essentially means that the buffer is growing exponentially pretty fast
and - according to KDE heaptrack - my system allocated ~9.5GB after 20s
while running some tests.

I added an exit-condition to the loop traversing through
inherit-subexpressions to avoid that. Checking for an "unexpected EOF"
is actually sufficient here:

• There's either a ; later in the expression causing the loop to
  terminate and causing an actual "unexpected token" error then.

• Otherwise, parse_val will go through the tokens until a matching
  semicolon is found (which is not the case) and then reach the end of
  the file. In that case, unexpected EOF is returned by parse_val.

[1] nix-community/rnix-lsp#33
[2] https://www.cs.cornell.edu/courses/cs3110/2011sp/Lectures/lec20-amortized/amortized.htm

[Ma27]

@fufexan btw would you mind building your rnix-lsp with this branch of rnix-parser and check for a few days if the issue is really gone for you? :)

[fufexan]

@Ma27 I would, I'm just not sure how to build it, as I haven't worked with rust/cargo before, so I've got no clue how to replace the default rnix package with the branch.

[Ma27]

Oh right, sorry!

The thing is, naersk doesn't seem to support Git repositories as crate overrides, so I took a bit more manual way myself:

• You'll basically need a Nix expression like https://gitlab.com/Ma27/nvim.nix/-/blob/master/flake.nix#L146-159
• And the patch which pins rnix-parser to this branch: https://gitlab.com/Ma27/nvim.nix/-/blob/master/rnix-fix.patch

[aaronjanse]

Whoa! Thank you so much @Ma27 @fufexan!

Everything here LGTM, although I haven't yet had a chance to run the code. Tomorrow morning I'll try to write a flake to make it easy to build rnix-lsp with a custom rnix-parser

[aaronjanse]

This should build rnix-parser, using this PR:

nix build git+https://gist.github.com/aaronjanse/0b6145784fcb1aa6ee1419248ae53872

[Ma27]

@fufexan did you have a chance to test this? :)

[fufexan]

@Ma27 yes, all works fine. Haven't had a memleak since switching :)

[aaronjanse]

This is great, and the code looks good to me. Thank you @Ma27!