Tracking Issue: Language Extensions

[maxbrunsfeld]

Problem

Right now, all of the languages that Zed supports are bundled into the binary. This means that users cannot add support for their own languages. It also means that the more languages Zed supports, the longer our compile time becomes.

Proposed Plan

Zed's first form of user-installed extensions will be language extensions. These plugins will have a directory structure similar to the per-language folders in Zed's current languages folder. In addition to the TOML config, and the query files, they will also contain a WASM build of the Tree-sitter parser.

In order to promote sharing of plugins, we'll create a new GitHub repository in the Zed organization that contains some directory of all known plugins. We'll add functionality to zed.dev so that when editing an unsupported language, Zed will be able to query for any known plugins that add support for that file extensions, and automatically download and install them.

Steps

Tree-sitter

• Enhance the Tree-sitter library to support loading Wasm files (tree-sitter#1864, tree-sitter#2840)
• Improve tooling for compiling Tree-sitter parsers to WASM (tree-sitter#2897)
  • Indicate when external scanners are using unsupported libc functions
  • Add any additional libc functions needed by external scanners in community parsers
• Land upstream wasmtime changes, so that we can publish a new Tree-sitter CLI to crates.io, and people can easily use the new WASM build tooling (wasmtime#7837, wasmtime#7872)

Zed Extension Support

• Allow Zed to load plugins on disk (Lay some groundwork for language plugins #3837)
• Figure out what macOS entitlements must be added to the application bundle so that Wasmtime's JIT compilation will be allowed, and the app won't be killed
• Improve how extensions are loaded from disk (Add logic for managing language and theme extensions #7467)
  • avoid synchronous IO
  • plugins should take precedence over built-in languages
  • when a plugin changes, just reload the language from that one plugin, not all languages

Convenient Extension Installation

• Create an extensions repository to which we can add submodules, in order to automatically package community extensions and provide them to Zed.
• Add a zed.dev API for searching extensions by name
• Add an extension installation view to Zed (Add an extensions installation view #7689)
• One by one, verify that all of these languages successfully build and load via the wasm system
• Add a zed.dev API that searches extensions by file extensions
• Add logic for automatically searching for extensions when editing files with unsupported extensions

Language Server Extensions

• Allow Zed to load WebAssembly extensions that provide logic for installing and running language servers (Add initial support for defining language server adapters in WebAssembly-based extensions #8645)
• Allow extensions to format LSP data (completions, workspace symbols, diagnostics)
• Make a smooth workflow for developing extensions locally
• Handle LSP extensions in the extensions-packaging CI job
• Publish an initial release of the zed-extension-api crate to crates.io

[foxkdev]

@maxbrunsfeld what are difference between plugins and extensions?

Other Question, this topic:

• Add logic in Zed to automatically download and install these extensions
  Design included? Same VSCODE but with Zed Style?
  

Can I try to implement this? Im know we need API definition but I can fake and when have API, we modify definition.
Im assume API will have this similar structure:

{
    "extensions": [
        {
            "name": "Test-Extension",
            "repostiory": "https://github.com/zed/extension/test-extension",
            "version": "1.0.0",
            "description": "This is a test extension",
            "author": "Zed"
        }
    ]
}

Give me feedback and if you want I can try to implement.

[maxbrunsfeld]

@foxkdev My plan is to start with the backend part of this API. We'll be adding REST APIs to zed.dev for this. I'm not sure exactly how the API will look, and we're going to be making changes rapidly, but you are welcome to take a shot at.

In general, we'll be working in the language extensions channel in Zed. You're welcome to join.

[claytonrcarter]

I was just playing w/ creating an extension for Laravel Blade files. It went OK following the example at https://github.com/zed-extensions/nu and also the stuff in https://github.com/zed-industries/zed/tree/main/crates/zed/src/languages. I did get it working, at least to the extent that that tree-sitter parser allowed, but I have a few quick comments:

• debug output was non-existent in the log, which made it hard to figure out what I was doing wrong when things weren't working
  • in my case, I had a few things (wrong filename for the wasm blob, various small issues w/ the injections and highlights files) that I was surpised didn't cause any error/warnings in the logs
• it's not clear how to tell the editor that one language should "beat" another, or how to disambiguate file extensions
  • in the case of Blade, the extension is .blade.php, and Zed is treating such files as PHP (which makes sense), and it wasn't clear how to tell it other wise
  • selecting "Laravel Blade" in the language selector works fine, and using a bogus but non-conflicting extension will also work fine
• prettier support ... I see how to configure the prettier parser in the toml, but it looks like the only way to define which prettier plugin to load is in a Rust file
  • Example w/ Svelte: parser in toml and plugin in Rust
  • maybe this will come w/ LSP support in extensions, but – at least for something like Blade – there are a few prettier plugins, but LSPs not so much

Thank you! It was fun playing with this and getting it set up.

[brookback]

Seconded on Clayton's comment above regarding debug logs. I'm trying — for fun — to add support for SQL grammar (based on https://github.com/DerekStride/tree-sitter-sql), but I can't for my life get it to work 😄

I've mimicked a bunch of existing Zed language extensions, but it's very hard without any logs telling me why it's not working properly. The repo: https://github.com/johanbrook/zed-sql

My woes

"SQL" shows up in the language selector when I bring it up manually, but it's not:

• associating my SQL language with .sql files automatically, even though I've configured it to.
• picking up any syntax highlighting or grammar. I might've messed up the WASM generation part locally, but who knows 🔮

[joshrutkowski]

What work is remaining to support language servers via extensions? I see my PR #8384 was closed in favor of this ongoing work - looking at where I can help out 😄

[jianghoy]

For anyone who's late to the party, right now the language folder is this: https://github.com/zed-industries/zed/tree/main/crates/languages/src (correct me if I'm wrong)

[jianghoy]

I was just playing w/ creating an extension for Laravel Blade files. It went OK following the example at https://github.com/zed-extensions/nu and also the stuff in https://github.com/zed-industries/zed/tree/main/crates/zed/src/languages. I did get it working, at least to the extent that that tree-sitter parser allowed, but I have a few quick comments:

• debug output was non-existent in the log, which made it hard to figure out what I was doing wrong when things weren't working

  • in my case, I had a few things (wrong filename for the wasm blob, various small issues w/ the injections and highlights files) that I was surpised didn't cause any error/warnings in the logs

• it's not clear how to tell the editor that one language should "beat" another, or how to disambiguate file extensions

  • in the case of Blade, the extension is .blade.php, and Zed is treating such files as PHP (which makes sense), and it wasn't clear how to tell it other wise
  • selecting "Laravel Blade" in the language selector works fine, and using a bogus but non-conflicting extension will also work fine

• prettier support ... I see how to configure the prettier parser in the toml, but it looks like the only way to define which prettier plugin to load is in a Rust file

  • Example w/ Svelte: parser in toml and plugin in Rust
  • maybe this will come w/ LSP support in extensions, but – at least for something like Blade – there are a few prettier plugins, but LSPs not so much

Thank you! It was fun playing with this and getting it set up.

Based on my findings in the above issue, I believe today Zed doesn't reconcile multiple language matches. I propose simply reconciling using the length of longest longest-matched suffix of each language extension.

[Feel-ix-343]

Note for linux users: Make sure you have a xdg FileChooser, else add dev extension will not work. Hyprland does not have a file chooser in xdg-desktop-portal; you must install one.

Also, this error is not too apparent and I needed to modify the source code to log it. Zed should log errors like this? relevant is linux/platform.rs ~line 230

[maxbrunsfeld]

Language extensions work now.

[gopeter]

Before opening a new issue: does Zed support multiple LSPs at once for one language? I came across that TypeScript autocompletion does not work when using the Biome extension: biomejs/biome-zed#10

[maxdeviant]

Before opening a new issue: does Zed support multiple LSPs at once for one language? I came across that TypeScript autocompletion does not work when using the Biome extension: biomejs/biome-zed#10

There are currently some issues when there are two language servers running for a given language.

I'm looking to address that with #10906.