Explore using tree sitter for syntax highlighting

[alexr00]

Goals of the explorations:

• Get Aiday's exploration up and running again: [Exploration] Tree-sitter tokenization exploration (Fixes #161256) #161479
• Come up with ideas for a contribution model
• Confirm performance findings from [Exploration] Tree-sitter tokenization exploration (Fixes #161256) #161479 and understand incremental highlighting

[alexr00]

Some notes on incremental parsing:

• Per Can web-tree-sitter actually do async parsing? tree-sitter/tree-sitter#2118, it looks like we can set a timeout so that each call to parse times-out, with the parsing progress preserved for the next call to parse.
• I've confirmed that this is the case; however, it appears to be broken in the latest version of web-tree-sitter. Going back to 0.22.2 works (latest is 0.22.5).
• There does not appear to be any significant time penalty for setting a timeout.

@hediet FYI.

[alexr00]

Opened tree-sitter/tree-sitter#3341 for the above issue.

[alexr00]

After some discussion, we've decided to first understand the performance of using tree-sitter to parse files. #213565 contains that work.

[alexr00]

We also need new internal editor API to write a more optimal parse function, which @rebornix may help with.

[alexr00]

Progress in July:

• Created a build pipeline for the relevant wasm files (tree-sitter and typescript right now): https://github.com/microsoft/vscode-tree-sitter-wasm
• Published first npm package with said wasm files: https://www.npmjs.com/package/@vscode/tree-sitter-wasm
• Incorporated that package in Add telemtry for how long it takes to parse files with tree-sitter #213565, which is almost ready to merge, but we'll hold off until after endgame as there's no need to rush it in before release.

[alexr00]

Updates (though I didn't get to work on this October):

• We have syntax highlighting tests for tree sitter
• Typescript is still the only supported language
• Lots of small typescript improvements

For November, the main plan is performance.

[Willisplayed]

That's along of junk

[Plixo2]

Can we get an update on this?