Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add telemtry for how long it takes to parse files with tree-sitter #213565

Merged
merged 43 commits into from
Jul 23, 2024
Merged

Add telemtry for how long it takes to parse files with tree-sitter #213565

Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
e1ea829
Make space for tree sitter
alexr00 May 7, 2024
150354b
Add the tree sitter wasm file
alexr00 May 13, 2024
1ee4fe2
Very naive tree-sitter syntax highlighting for html, with a layer bre…
alexr00 May 21, 2024
3614235
Update tree when content changes
alexr00 May 22, 2024
2703b6b
WIP for making abstract tokens class
alexr00 May 22, 2024
edde184
Handle theme changes
alexr00 May 22, 2024
5c20e64
Replace entire text model value with parse callback
alexr00 May 22, 2024
4cd3e08
Perf improvements
alexr00 May 23, 2024
fef5067
Add tree-sitter-typescript
alexr00 May 23, 2024
5c65dfa
Add typescript + better initial parsing
alexr00 May 24, 2024
c372e40
Refactor into tree parsing service and fix flaw in parse callback
alexr00 May 27, 2024
d411bfc
Remove things that aren't the parser service
alexr00 May 27, 2024
7c0e5d5
Add yielding
alexr00 May 27, 2024
e32a24d
Remove changes that aren't required for PR
alexr00 May 27, 2024
7f0678a
Remove more file changes
alexr00 May 27, 2024
eb1bba2
Reduce yield to 50 ms
alexr00 Jun 18, 2024
d56de42
Fix incremental parsing
alexr00 Jun 18, 2024
194527c
Merge branch 'main' into alexr00/treeSitterParserService
alexr00 Jun 18, 2024
df28801
Try update node-abi
alexr00 Jun 18, 2024
85ba149
Revert "Try update node-abi"
alexr00 Jun 18, 2024
70e6fdd
Update text buffer chunk api
rebornix Jun 18, 2024
b728bd9
fix build
rebornix Jun 18, 2024
eb265f5
Merge pull request #215471 from microsoft/rebornix/right-cricket
rebornix Jun 18, 2024
0563837
Remove tree-sitter dependency
alexr00 Jun 19, 2024
37f5af6
Merge branch 'main' into alexr00/treeSitterParserService
alexr00 Jul 8, 2024
ab57877
Merge branch 'main' into alexr00/treeSitterParserService
alexr00 Jul 9, 2024
c28ede5
Adopt new, as yet unpublished, `@vscode/tree-sitter-wasm` package
alexr00 Jul 10, 2024
9747616
Use published `@vscode/tree-sitter-wasm` package
alexr00 Jul 11, 2024
8c7116c
Break `TreeSitterTree` and `TreeSitterParserService` into better pieces
alexr00 Jul 12, 2024
3a9a103
Merge remote-tracking branch 'origin/main' into alexr00/treeSitterPar…
alexr00 Jul 15, 2024
7148842
Fix tests
alexr00 Jul 15, 2024
2c2e321
Remove unneeded import
alexr00 Jul 15, 2024
028c770
Fix missing tree-sitter-wasm in web and remote
alexr00 Jul 16, 2024
539a273
Make package.jsons match
alexr00 Jul 16, 2024
e6c2529
Add @vscode/tree-sitter-wasm to web loader config
alexr00 Jul 17, 2024
4f98572
Merge branch 'main' into alexr00/treeSitterParserService
alexr00 Jul 17, 2024
04232a4
Try using importAMDNodeModule
alexr00 Jul 17, 2024
0ca55da
PR feedback
alexr00 Jul 18, 2024
6435924
Add race condition test for changing language while loading language
alexr00 Jul 18, 2024
7e82f06
Use same timeout
alexr00 Jul 18, 2024
13a6a3b
Queue content changes
alexr00 Jul 18, 2024
d08bbb1
Remove override dispose
alexr00 Jul 19, 2024
26edd5c
Move queue into TreeSitterTree
alexr00 Jul 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .eslintrc.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -755,7 +755,8 @@
"vs/base/~",
"vs/base/parts/*/~",
"vs/platform/*/~",
"vs/editor/~"
"vs/editor/~",
"@vscode/tree-sitter-wasm" // node module allowed even in /common/
]
},
{
Expand Down
1 change: 1 addition & 0 deletions build/.webignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ vscode-textmate/webpack.config.js

# This makes sure the model is included in the package
!@vscode/vscode-languagedetection/model/**
!@vscode/tree-sitter-wasm/wasm/**

# Ensure only the required telemetry pieces are loaded in web to reduce bundle size
@microsoft/1ds-core-js/**
Expand Down
1 change: 1 addition & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@
"@vscode/spdlog": "^0.15.0",
"@vscode/sqlite3": "5.1.6-vscode",
"@vscode/sudo-prompt": "9.3.1",
"@vscode/tree-sitter-wasm": "^0.0.1",
"@vscode/vscode-languagedetection": "1.0.21",
"@vscode/windows-mutex": "^0.5.0",
"@vscode/windows-process-tree": "^0.6.0",
Expand Down
1 change: 1 addition & 0 deletions remote/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
"@vscode/proxy-agent": "^0.22.0",
"@vscode/ripgrep": "^1.15.9",
"@vscode/spdlog": "^0.15.0",
"@vscode/tree-sitter-wasm": "^0.0.1",
"@vscode/vscode-languagedetection": "1.0.21",
"@vscode/windows-process-tree": "^0.6.0",
"@vscode/windows-registry": "^1.1.0",
Expand Down
1 change: 1 addition & 0 deletions remote/web/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
"@microsoft/1ds-core-js": "^3.2.13",
"@microsoft/1ds-post-js": "^3.2.13",
"@vscode/iconv-lite-umd": "0.7.0",
"@vscode/tree-sitter-wasm": "^0.0.1",
"@vscode/vscode-languagedetection": "1.0.21",
"@xterm/addon-clipboard": "0.2.0-beta.34",
"@xterm/addon-image": "0.9.0-beta.51",
Expand Down
5 changes: 5 additions & 0 deletions remote/web/yarn.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,11 @@
resolved "https://registry.yarnpkg.com/@vscode/iconv-lite-umd/-/iconv-lite-umd-0.7.0.tgz#d2f1e0664ee6036408f9743fee264ea0699b0e48"
integrity sha512-bRRFxLfg5dtAyl5XyiVWz/ZBPahpOpPrNYnnHpOpUZvam4tKH35wdhP4Kj6PbM0+KdliOsPzbGWpkxcdpNB/sg==

"@vscode/tree-sitter-wasm@^0.0.1":
version "0.0.1"
resolved "https://registry.yarnpkg.com/@vscode/tree-sitter-wasm/-/tree-sitter-wasm-0.0.1.tgz#ffb2e295a416698f4c77cbffeca3b28567d6754b"
integrity sha512-m0GKnQ3BxWnVd+20KLGwr1+Qvt/RiiaJmKAqHNU35pNydDtduUzyBm7ETz/T0vOVKoeIAaiYsJOA1aKWs7Y1tA==

"@vscode/vscode-languagedetection@1.0.21":
version "1.0.21"
resolved "https://registry.yarnpkg.com/@vscode/vscode-languagedetection/-/vscode-languagedetection-1.0.21.tgz#89b48f293f6aa3341bb888c1118d16ff13b032d3"
Expand Down
5 changes: 5 additions & 0 deletions remote/yarn.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,11 @@
mkdirp "^1.0.4"
node-addon-api "7.1.0"

"@vscode/tree-sitter-wasm@^0.0.1":
version "0.0.1"
resolved "https://registry.yarnpkg.com/@vscode/tree-sitter-wasm/-/tree-sitter-wasm-0.0.1.tgz#ffb2e295a416698f4c77cbffeca3b28567d6754b"
integrity sha512-m0GKnQ3BxWnVd+20KLGwr1+Qvt/RiiaJmKAqHNU35pNydDtduUzyBm7ETz/T0vOVKoeIAaiYsJOA1aKWs7Y1tA==

"@vscode/vscode-languagedetection@1.0.21":
version "1.0.21"
resolved "https://registry.yarnpkg.com/@vscode/vscode-languagedetection/-/vscode-languagedetection-1.0.21.tgz#89b48f293f6aa3341bb888c1118d16ff13b032d3"
Expand Down
1 change: 1 addition & 0 deletions src/bootstrap-window.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,7 @@ const isESM = false;
// using a fallback such as node.js require which does not exist in sandbox
const baseNodeModulesPath = isDev ? '../node_modules' : '../node_modules.asar';
loaderConfig.paths = {
'@vscode/tree-sitter-wasm': `${baseNodeModulesPath}/@vscode/tree-sitter-wasm/wasm/tree-sitter.js`,
'vscode-textmate': `${baseNodeModulesPath}/vscode-textmate/release/main.js`,
'vscode-oniguruma': `${baseNodeModulesPath}/vscode-oniguruma/release/main.js`,
'vsda': `${baseNodeModulesPath}/vsda/index.js`,
Expand Down
Loading
Loading