"source_file" allows "BLANK" to follow "_top_level_declaration"

[berchn]

source_file allows BLANK to follow _top_level_declaration in src/grammar.json. This is resulting in an AST that doesn't appear to be valid. The following code parses without error or the addition of any zero-width tokens, but it does not compile:

package a package b package c package d

This is due to the optional(terminator) in the relevant section of the source_file rule in grammar.js. Is this intended?

[adonovan]

The optionality should be removed here, but I think it might require additional handling to ensure that EOF inserts a final newline (terminator):

tree-sitter-go/grammar.js

Line 107 in 07d7228

seq($._top_level_declaration, optional(terminator)),

I suspect in practice it's less trouble to make the parsers overgenerous in what they accept than to have to analyze the dark corners perfectly (where compilers often have bugs too).