Include generated files in repo

[diktomat]

I understand your reasons for not including them, however, lots of editors configure tree-sitters by git url, which of course isn't possible if it's not in the repo. Examples being Helix (issue) or Nvim, both supporting local files too, but that of course is an extra step to take and keep up to date and can't be included in default configurations.

[alex-pinkus]

Nvim seems to do ok with things as they are, and this is pretty clearly the direction that the tree-sitter folks want to take things. I’d rather invest the effort in getting Helix to a state where it can handle this. Is there a good explanation somewhere of what’s still needed for that?

[alex-pinkus]

As a side note, nvim-treesitter’s ability to regenerate parsers from source also lets them take advantage of query optimizations that require an ABI bump, which I implemented in tree-sitter specifically to benefit Swift. If we pregenerated the parser files we wouldn’t be able to take advantage of those until every consumer was on ABI 14.

[the-mikedavis]

For Helix to not depend on generated parser files checked into source: helix-editor/helix#1970

Generating the parser files with tree-sitter-cli is not a good fit for Helix. I think a runtime or build-time dependency on tree-sitter-cli flies in something like nvim-treesitter because tree-sitter is the main feature, but for tools where tree-sitter is an implementation detail (semgrep and difftastic for example), it's an undue hassle for users and packagers.

On the one hand I detest checking in generated src and I agree tree-sitter consumers should not assume they're checked in. OTOH it's a pragmatic and low-cost way to make it easy for the majority of tree-sitter consumers to package the grammar. It also doesn't stop anyone from using a non-default ABI: eager consumers would just need to re-generate the parser.

[alex-pinkus]

That’s a fair point.

To me the biggest concern with the generated files is that they make pull requests more likely to have merge conflicts. Could we maybe compromise by having a separate branch with generated source? GitHub Actions could trigger an automatic checkin every time we publish a new release. That way the diffs would stay clean and there wouldn’t be too many new large deltas in the repo, but the system would still look like a git-based tree-sitter parser.

[alex-pinkus]

All right, this has now been done! The with-generated-files branch of tree-sitter-swift gets automatically updated every time I publish a new release (actually, any new tag). I also published 0.2.0 to seed it with an initial set of static files. @d12bb @the-mikedavis FYI